Bilingual Synchronization: Restoring Translational Relationships with Editing Operations (2022)

Abstract: Machine Translation (MT) is usually viewed as a one-shot process that generates the target language equivalent of some source text from scratch. We consider here a more general setting which assumes an initial target sequence, that must be transformed into a valid translation of the source, thereby restoring parallelism between source and target. For this bilingual synchronization task, we consider several architectures (both autoregressive and non-autoregressive) and training regimes, and experiment with multiple practical settings such as simulated interactive MT, translating with Translation Memory (TM) and TM cleaning. Our results suggest that one single generic edit-based system, once fine-tuned, can compare with, or even outperform, dedicated systems specifically trained for these tasks.

## introduction neural machine translation (nmt) systems have made tangible progress in recent years @xcite , as they started to produce usable translations in production environments. nmt is generally viewed as a one-shot activity process in autoregressive approaches, which generates the target translation based on the sole source side input. recently, non-autoregressive machine translation (nat) models have proposed to perform iterative refinement decoding @xcite @xcite , where translations are generated through an iterative revision process, starting with a possibly empty initial hypothesis. this paper focuses on the revision part of the machine translation (mt) process and consider bilingual synchronization (bi-sync), which we define as follows: given a pair of a source (f ) and a target (ẽ) sentences, which may or may not be mutual translations, the task is to compute a revised version e of ẽ, such that e is an actual translation of f . this is necessary when the source side of an existing translation is edited, requiring to update the target and keep both sides synchronized. bi-sync subsumes standard mt, where the synchronization starts with an empty target (ẽ = []). other interesting cases occur when parts of the initial target can be reused, so that the synchronization only requires a few changes. bi-sync encompasses several tasks: synchronization is needed in interactive mt (imt, @xcite and bilingual editing @xcite , with ẽ the translation of a previous version of f ; in mt with lexical constraints @xcite , where ẽ contains target-side constraints @xcite ; in translation memory (tm) based approaches @xcite , where ẽ is a tm match for a similar example; in automatic post-editing (ape) @xcite , where ẽ is an mt output. we consider here several implementations of sequence-to-sequence models dedicated to these situations, contrasting an autoregressive model with a non-autoregressive approach. the former is similar to @xcite , where the source sentence and the initial translation are concatenated as one input sequence; the latter uses the levenshtein transformer (levt) of @xcite . we also study various ways to generate appropriate training samples (f , ẽ, e). our experiments consider several tasks, including tm cleaning, which attempts to fix and synchronize noisy segments in a parallel corpus. this setting is more difficult than bi-sync, as many initial translations are already correct and need to be left unchanged. our results suggest that one single ar system, once fine-tuned, can favorably compare with dedicated systems for each of these tasks. to recap, our main contributions are (a) the generalization of several tasks subsumed by a generic synchronization objective, allowing us to develop a unified perspective about otherwise unrelated subdomains of mt; (b) the design of a pas cela cela pas . top 5 sampling constrained decoding cela n' arrivera pas . ## translating with translation memories as explained above, bi-sync encompasses examplebased mt, whereby an existing similar translation retrieved from a tm is turned into an adequate translation of the source. edit-mt actually uses the same architecture as the retrieval-based models of @xcite ; @xcite . in this section, we study the performance of our synchronization models in this practical scenario. ## parallel corpus cleaning our model restores synchronization between a pair of sentences. this is also useful for parallel tm cleaning tasks. given a source sentence f and a possibly incorrect translation ẽ, we want to detect non-parallelism and to perform appropriate fixes. we study how edit-mt fares with this new problem on two publicly available datasets: first on the semeval 2012&3 task 8: cross-lingual textual entailment @xcite @xcite ), foot_13 then with the opensubtitles corpus @xcite . ## conclusion this work introduced bi-sync, the task of generating translations of a source sentence by editing a related target sentence. we have proposed various ways to create artificial initial translations for different editing types, that are needed for training. we have explored both autoregressive and non-autoregressive architectures, observing experimentally that our autoregressive edit-mt model trained with artificial triplets performs bilingual resynchronization tasks in several real world scenarios. edit-mt can also be quickly adapted to retrieval-based mt tasks, where we compared favorably to dedicated models. finally, edit-mt can also fix tms by detecting parallel sentences and correct imperfect translations without adaptation. another application that we wish to explore is ape. our nat approach edit-levt is lagging behind edit-mt. in the future, we would like to explore more nat systems, which are computationally faster, and improve their performance. we intend to consider training curriculums and to modify the levt model to better fit the bi-sync task. we would also like to study ways to reduce the load of fully re-decoding the input sequence, especially when small changes, that need to be reproduced in the target, are iteratively applied to the source sentence.