E nglish-to- J apanese Diverse Translation by Combining Forward and Backward Outputs (2020)

Abstract: We introduce our TMU system that is submitted to The 4th Workshop on Neural Generation and Translation (WNGT2020) to English-to-Japanese (En→Ja) track on Simultaneous Translation And Paraphrase for Language Education (STAPLE) shared task. In most cases machine translation systems generate a single output from the input sentence, however, in order to assist language learners in their journey with better and more diverse feedback, it is helpful to create a machine translation system that is able to produce diverse translations of each input sentence. However, creating such systems would require complex modifications in a model to ensure the diversity of outputs. In this paper, we investigated if it is possible to create such systems in a simple way and whether it can produce desired diverse outputs. In particular, we combined the outputs from forward and backward neural translation models (NMT). Our system achieved third place in En→Ja track, despite adopting only a simple approach.

## related work several models have been proposed to generate diverse decoding outputs for different tasks. for example, @xcite proposed diverse paraphrase generation by introducing rewriting patterns into the decoder of the encoder-decoder model. @xcite proposed diverse beam search algorithm for decoding diverse sequences. they describe beam search as an optimization problem and augment the objective with a diversity term. they encouraged diversity between beams at each step by rewarding each group for spending its beam budget to explore different parts of the output space rather than repeatedly chasing sub-optimal beams from prior groups. they report their results on image captioning, visual question generation, and mt tasks. @xcite generated diverse translations by conditioning sentence generation with the sentence codes. they explored two methods: (a) semantic coding model which extracted sentence codes from unsupervisedly learned semantic information and (b) syntactic coding model which derived the sentence codes from the parse trees produced by a constituency parser. @xcite proposed the use of over-sampling followed by postdecoding clustering to remove similar sequences. they evaluated several techniques on an openended dialog task and image captioning task. these works introduce different complex modifications to the model in order to achieve diversity while generating the output. however, in this paper, we show how to simply generate diverse outputs. ## conclusion in this paper, we introduced our system submitted to wngt2020 shared task to @xmath0. we have shown that even a simple method which uses only forward and backward models' outputs can generate a variety of translations while maintaining original meaning and grammaticality. in future, we plan to compare our system with existing systems that perform different types of language generation. in addition, we will investigate the impact of l2r and r2l models to the diverse output in depth.