B itext E dit: Automatic Bitext Editing for Improved Low-Resource Machine Translation (2022)

Abstract: Mined bitexts can contain imperfect translations that yield unreliable training signals for Neural Machine Translation (NMT). While filtering such pairs out is known to improve final model quality, we argue that it is suboptimal in low-resource conditions where even mined data can be limited. In our work, we propose instead, to refine the mined bitexts via automatic editing: given a sentence in a language x_f , and a possibly imperfect translation of it \mathbf{x_e} , our model generates a revised version x_f' or x_e' that yields a more equivalent translation pair (i.e., < x_f, x_e' > or < x_f', x_e >). We use a simple editing strategy by (1) mining potentially imperfect translations for each sentence in a given bitext, (2) learning a model to reconstruct the original translations and translate, in a multi-task fashion. Experiments demonstrate that our approach successfully improves the quality of CCMatrix mined bitext for 5 low-resource language-pairs and 10 translation directions by up to 8 BLEU points, in most cases improving upon a competitive translation-based baseline.

## introduction neural machine translation (nmt) for lowresource languages is challenging due to the scarcity of bitexts, i.e., translated text in two languages @xcite . models are often trained on heuristically aligned @xcite @xcite or automatically mined data @xcite , which can be low quality @xcite . this data can include errors that range from small meaning differences in sentences that overlap in content to major differences that yield completely incorrect translations and random noise, e.g., empty sequences, text in the wrong language, non-linguistic content, among others. ## approach: bitextedit we frame bitext refinement as an editing task (i.e, bitextedit) that takes two input sentences: a sentence x f in language f and a sentence x e in language e, and aims at editing one of them (i.e., it outputs x f or x e ) with the goal of yielding a more equivalent translation pair (i.e., <x f , x @xmath0, x @xmath1. figure 2 gives an overview of our approach while below we describe the bitext refinement model ( §3.1) and the curation of data needed to train our model based on bitext mining ( §3.2). ## experimental setting bitexts we focus on ccmatrix data for two main reasons: a) it constitutes the only large-scale available resource for a lot of low-resource language pairs and b) recent efforts of auditing this corpus raise concerns regarding the quality of mined bitext of low-resource pairs. ccmatrix is mined using laser embeddings following the max-strategy approach: a margin score is computed for all monolingual sentences in two languages, then the union of forward and backward candidates is build and pairs that score above a pre-defined threshold are treated as translations. @xcite set the threshold globally for all languages at 1.06. our primary goal is to explore whether bitexts that are typically discarded by filtering can be refined by our model and thus benefit low-resource nmt. for this purpose, we define two pools of cc-matrix data: pool a corresponds to ccmatrix data with laser scores greater than 1.06, while pool b contains bitexts with scores lower than 1.06 and greater than 1.05. the latter threshold is primarily chosen since ccmatrix bitexts is only available above this value. editing bitexts with even smaller scores is an interesting area for future work. training data our models are trained based on procedures described in §3.2, where we use pool a to seed the generation of noised training samples x f and x e . we mine k samples x f for each x e and k samples x e for each x f , respectively. we set k to 4 and include detailed statistics in appendix f. ## analysis we now turn into analysis with a focus on understanding the broader space where bitextedit can be applied. we experiment with scaling-up bitext refinement to higher-resource settings in §6.1, we perform qualitative analysis on the edited bitexts in §6.2, and quantitative analysis on the types and intensity of edits in different corpora in §6.3. ## conclusion we introduce an alternative approach for bitext quality improvement that we show is better suited for low-resource language pairs. instead of filtering out imperfect translation references that result from automatic bitext mining, we instead edit them with the goal of improving their quality. our editing models are trained using only synthetic supervision, which can be gathered at scale for any language pair that support bitext mining. extensive quantitative analysis suggests that our approach successfully improves bitext quality for a variety of language-pairs and different resource conditions. furthermore, extrinsic experiments on 10 low-resource nmt tasks suggest that bitext refinement constitutes a successful approach to improving nmt translation quality in low data regimes. those findings highlight the importance of the good quality bitexts in scenarios where large quantities cannot be guaranteed and motivate future research on improving lowresource nmt further.