Monotonic Simultaneous Translation with Chunk-wise Reordering and Refinement (2021)

Abstract: Recent work in simultaneous machine translation is often trained with conventional full sentence translation corpora, leading to either excessive latency or necessity to anticipate as-yet-unarrived words, when dealing with a language pair whose word orders significantly differ. This is unlike human simultaneous interpreters who produce largely monotonic translations at the expense of the grammaticality of a sentence being translated. In this paper, we thus propose an algorithm to reorder and refine the target side of a full sentence translation corpus, so that the words/phrases between the source and target sentences are aligned largely monotonically, using word alignment and non-autoregressive neural machine translation. We then train a widely used wait-k simultaneous translation model on this reordered-and-refined corpus. The proposed approach improves BLEU scores and resulting translations exhibit enhanced monotonicity with source sentences.

## introduction simultaneous interpretation is widely used in various scenarios such as cross-lingual communication between international speakers, international summits, and streaming translation of a live video. simultaneous interpretation has a latency advantage over conventional full-sentence translation, i.e. offline translation, as it requires only partial sequence to start translating. however, as the source and target languages differ in word orders, there is a difficulty in simultaneous interpretation that does not exist in offline translation which translates only after the whole source sentence is received. for example, when dealing with language pairs that significantly differ in word order (e.g., between sov language and svo language), an interpreter figure 1 : an example illustration of monotonic reordering and refinement for simultaneous translation may not receive sufficient information with partial sequence to start generating a translation that respects the natural order of the target language. one of the approaches to address this problem is to perform anticipation foot_0 . note that the nature of anticipation relies on interpreters' assumptions and the anticipation may provide incorrect translations. alternatively, human interpreters strategically resort to producing monotonic translations that follow the word order of the source sequence @xcite . to illustrate the differences between the two strategies, the example in figure 1 may be referred to. of the two targets, offline target y respects the target language order and an online target ŷ roughly follows the source word ordering. successful anticipation in figure 1 's case would be to predict the initial words in y (i was a "frog in a well") before receiving the full x. this would pose difficulty even to professional translators as all the relevant information is in the latter part of the x (ᄌ ᅥᄂ ᅳ ᆫi /"ᄋ ᅮᄆ ᅮ ᆯ a well / ᄋ ᅡ ᆫin / ᄀ ᅢᄀ ᅮᄅ ᅵ a f rog " ᄋ ᅧ ᆻᄉ ᅳ ᆸᄂ ᅵᄃ ᅡwas.). @xcite reports the success rate of human interpreters' anticipation attempts to be as low as 38.1% even though they make predictions based on pre-acquired domain knowledge. on the other hand, a monotonic approach would be to gen-erate an ŷ style translation -the grammaticality in the resulting sequence is sacrificed to translate only the received information. a similar case applies to simultaneous machine translation (simulmt) models, which start translating before a whole sentence is given. several studies @xcite @xcite often utilize offline full-sentence translation corpora to train simulmt models. offline full-sentence parallel corpora are expected to follow the natural order of languages and mostly contain source to offline-target pairs. naturally, when simulmt models are trained on these corpora, the models inevitably learn to perform anticipation. recent simulmt studies are focused on reducing anticipation @xcite or performing better anticipation @xcite ). on the contrary, studies on enabling monotonic translation in simulmt are scarcely available. recently, @xcite suggest utilizing pseudoreferences for monotonic translation. in this paper, we propose a paraphrasing method to generate a monotonic parallel corpus to allow a monotonic interpretation strategy in simulmt. our method consists of two stages. the method first chunks source and target sequences into segments and monotonically reorders the target segments based on source-target word alignment information (section 3.1). then, the reordered targets are refined to enhance fluency and syntactic correctness (section 3.2). to show the effectiveness of our method, we train wait-k models @xcite on the resulting monotonic parallel corpus of reordering-and-refinement. results show improvements in bleu scores over baselines and models producing monotonic translations. our main contributions are as follows: ## monotonicity analysis in this section, we analyze the degree of word order differences in multiple language pairs, i.e., the monotonicity in different language pairs. to measure the monotonicity, two rank correlation statistics are utilized: kendall's τ and spearman's ρ. the analyzed language pairs are: english-{korean, japanese, chinese, german, french}. according to @xcite , english is a headinitial language and korean and japanese are rigid head-final languages; korean and japanese are likely to exhibit extreme word order differences with english. german and chinese are considered a mixture of head-final and head-initial languages; they are likely to have word differences with english, but not as severe as korean or japanese. french is also head-initial, so english and french pair is likely to have similar word order. figure 2 show monotonicity measurements between english and five different languages which vary in monotonicity: english-german and english-french pairs show high monotonicity, while english-japanese and english-korean pairs show low monotonicity. lower monotonicity in language pairs presents higher difficulties for simulmt tasks. for example, wait-k algorithm only sees k + t source tokens to generate a target token at step t which could lead to unwanted anticipation. to avoid such anticipation, as we mentioned in section 1, human interpreters often provide a monotonic translation. in the same sense, we conjecture that promoting monotonicity in training corpora is beneficial for translation quality in simulmt. ## monotonic reordering and refinement in this section, we describe our proposed paraphrasing method of chunk-wise reordering and refinement to generate monotonic corpus for simulmt. given source @xmath0, x 2 , • • • , x |x| } and offline full sentence target @xmath1, y 2 , • • • , y |y| }, an alignment a is defined as a set of position pairs of x and y. first, in chunk-wise reordering phase, we generate source chunk set c x c @xmath2:p 1 ), (x p 1 +1:p 2 ), • • • , (x p k-1 +1:p k )}, ## related work due to word order differences between languages, simulmt training often face situations where anticipation is required. note that word order difference is observed to be problematic even for human interpreters @xcite . @xcite suggest using pseudo-references which involve utilizing wait-k inference output to limit "future anticipation" in training. @xcite utilize nmt teachers to implicitly embed future information in their simulmt students for better anticipation performance. @xcite study adaptive policy to tackle this problem -authors suggest an adaptive simulmt policy that dictate read/write actions based on whether "meaningful units" are fully formed with consumed input tokens. related work in the broader simulmt and para-phrasing domain is presented in appendix f. ## conclusion most of simulmt models are trained on offline translation corpora, which could lead to limitation in translation quality and achievable latency, especially in non-monotonic language pairs. to address this problem, we propose a reordering-andrefinement algorithm to generate monotonically aligned online target with nat model. we then train widely used wait-k simulmt models on this newly generated corpus. resulting models show bleu score improvement and significant enhancement on monotonicity in multiple language pairs.