Example-Based Machine Translation with a Multi-Sentence Construction Transformer Architecture (2023)

Abstract: Neural Machine Translation (NMT) has now attained state-of-art performance on large-scale data. However, it does not achieve the best translation results on small data sets. Example-Based Machine Translation (EBMT) is an approach to machine translation in which existing examples in a database are retrieved and modified to generate new translations. To combine EBMT with NMT, an architecture based on the Transformer model is proposed. We conduct two experiments respectively using limited amounts of data, one on an English-French bilingual dataset and the other one on a multilingual dataset with six languages (English, French, German, Chinese, Japanese and Russian). On the bilingual task, our method achieves an accuracy of 96.5 and a BLEU score of 98.8. On the multilingual task, it also outperforms OpenNMT in terms of BLEU scores.

## introduction an analogy is a relationship between four objects, a is to b as c is to d. studies on analogies have investigated their utility in different applications, like machine translation. solving analogies between sentences involves the task of generating an unknown d that satisfies an analogical equation a : b :: c : d , where a, b, and c are given. here is an example of a sentence analogy: he 's coming . : i am coming . :: formula (1) defines the notation of analogies between sentences in two languages. as instantiated in formula (2), the translation result for "i am eating an apple ." is "je manger une pomme .", which can be obtained through the reasoning process. a : b :: c : ## previous work and proposal to perform translation, @xcite proposed an approach to ebmt that considers a bilingual analogy across two languages. translations are made by transferring symbolic knowledge from the source language to the target language. in figure 1(a), the translation of "i am eating an apple ." is achieved by solving a bilingual analogy: i am coming . figure 1(b) outlines the indirect approach to ebmt. as previously shown in formula (2), previous research considered two monolingual analogies in two different languages that correspond to ## multi-sentence construction transformer architecture we propose a novel transformer structure that allows for multiple sentences to be inputted simultaneously, compared to the vanilla transformer's single-sentence input. concretely, this multisentence construction transformer architecture takes seven sentences a, b, c, d, a ′ , b ′ , c ′ as input to generate the target translation d ′ . rather than concatenating them into a single input, we employ seven distinct inputs, which allows each individual input to compute attention with the output. ## experiments and analysis to evaluate the performance of our proposed model, we compare its translation results to those of other methods. for the bilingual translation task from english to french, we use opennmt foot_3 @xcite and the method proposed by @xcite as baselines. for the multilingual translation task across six languages, we use opennmt only. the parameter settings for opennmt and our proposal are detailed in appendix a. ## conclusion we proposed a multi-sentence construction transformer architecture model to implement ebmt by analogy. our proposal outperformed the two baselines on the bilingual dataset, achieving a bleu score of 98.8 and an accuracy of 96.5. additionally, for the multilingual translation task across six languages, our proposed method produced significantly better results than opennmt.