Does partial pretranslation can improve low ressourced-languages pairs? (2022)

Abstract: We study the effects of a local and punctual pretranslation of the source corpus on the performance of a Transformer translation model. The pretranslations are performed at the morphological (morpheme translation), lexical (word translation) and morphosyntactic (numeral groups and dates) levels. We focus on small and medium-sized training corpora (50K 2.5M bisegments) and on a linguistically distant language pair (Japanese and French). We find that this type of pretranslation does not lead to significant progress. We describe the motivations of the approach, the specific difficulties of Japanese-French translation. We discuss the possible reasons for the observed underperformance.

## introduction there are many techniques to improve the performance of a neural translation system without changing the size of the training corpus, without increasing the computational power, and independently of the tuning of the translation system: enriching vocabulary embedding (e.g. @xcite ), injecting linguistic information (e.g. @xcite ), reordering (e.g. @xcite ), etc. these techniques are absolutely crucial for language pairs with few corpora, and when computing power is limited. we propose here to apply another technique, which to our knowledge has not been studied so far with neural translation. it consists in pretranslating short segments of the source corpus. we proceed with handmade rules and vocabulary translation lists. we will observe its effects on japanese-french, with several corpus sizes (50k-2.5m bisegments). this is indeed a language pair that remains poorly endowed with large, freely accessible and good quality corpora. the aim of pretranslation is to reduce the linguistic distance between the two languages and to facilitate learning. the advantage is that it can be applied even with limited knowledge about the translation rules between the two languages. in addition, building the pretranslation rule by hand does not necessitate annotated resources (that maybe do not exist). in section 2 we present the difficulties specific to french-japanese. in section 3 we describe the type of pretranslations we have carried out. in section 3, we describe an experimental setup used to evaluate the effects of those pretranslations. we will see in section 5 that it is difficult to correlate these results with the properties of the corpora. ## challenges of japanese-french translation japanese and french are known to be linguistically distant languages. we present here briefly the most notable points of divergence, which may impact their joint treatment. japanese and french use different writing systems. japanese uses about 2300 characters. french uses about 50 (including capitals). the two systems share only arabic numerals and some english words written in latin characters in japanese texts. the sharing of vocabulary is therefore of little interest when training a translation model. word formation in the two languages does not correspond. french is a richly inflected language. flexion concerns almost all parts of speech. on the contrary, japanese morphology varies only for a part of verbs and adjectives (native japanese lexical stratum). japanese is also considered an agglutinative language. with the same meaning, many expressions in japanese and french are not formed at the same level: expressions formed at the morphological level in japanese are formed at the syntactic level in french and vice versa (see example below "seem not to want to drink"; conj:conjugation suffix): rad conj conj conj conj drink want neg past seem sembl-ait ne pas vouloir boire rad conj aux aux v v seem past neg neg want drink word order is different at several levels. in writing, japanese is an sov language where the order is semi-free, with possible pragmatic effects. french is a svo language. word order also diverges within phrases. in japanese, in most cases, the complements (nominal, propositional) precede the head of the phrase. in french, they appear on either side of the head: a major source of difficulty in comparing or translating the two languages is the absence in japanese of many components that are obligatory in french. japanese uses few quantification marks (determiners etc.) and makes extensive use of bare nouns. a bare noun phrase will often have several possible translations in french (see also a discussion involving japanese and english in @xcite . many sentence components are elided in japanese. japanese is a pro-drop language. unlike in french, the place of absent components is not occupied by a pronoun. in addition, titles and press headlines have a specific syntax in japanese @xcite . constraints between distant structures exist in both languages but do not concern the same parts of speech. in french, distant words can share the same gender and number marks. this is the case of subject-verb agreement, for example. japanese is known to use floating quantifiers consisting of a numeral and a classifier. the choice of the classifier depends on the quantified noun. ## pretranslation applied here we study 5 levels of pretranslation. the pretranslations are applied recursively (the pretranslation of corpus i is added to that of corpus i -1). the idea is that a single modification cannot substantially improve the translation. we must therefore study an accumulation of pretranslations. an example of sentence pretranslation is given in the ## discussion as expected, pretranslation increases the proximity (measured in bleu, on bpe segmented corpus) between the japanese and french corpora (see tab. 5 ). for the smaller training corpora (50k bisegments), the progression is a little less than 1 point (knowing that in percentage, this represents 50%). the pud corpus shows the most notable progress (+1.63 points). however, it should be noted that, whatever the corpus, the proximity is low, with or without pretranslation (<3.80 bleu). the results do not show significant progress. in some cases, there is even a deterioration. nor is there a clear causal link between the (quantified) characteristics of the corpus and the results. compared to the size of the training corpus (50k, 500k or 2.5m bisegments), we observe a systematic but modest improvement for the 500k bisegment corpus. this behaviour is correlated with a very slight superiority of the vocabulary variety (tab.4). in other words, for this size of corpus and type of corpus, the greater variety of the corpus could improve the translation. concerning the test corpora, we observe better results for pud. again, in parallel, we note that the variety of vocabulary is slightly higher for pud (tab.4). moreover, if we observe the proportional difference of the number of words in japanese and french (# words ja -# words fr/ # words ja) we see that pud is close to train.2m5 (tab.7). we also observe a (very slightly) higher proximity between ja and fr (bleu) for pud (tab.5). we also repeated the c2 and c4 experiment with vocabulary sharing (word number for bpe segmentation is set at 16k). the results are slightly lower than with the separated vocabularies. this can be explained by the low number of common word strings, even after pre-translation. vocabulary sharing does not improve the results after pre-translation. it is difficult to establish a causal relationship between the (quantifiable) characteristics of the corpora and the results. indeed, it can be observed that corpus features such as proximity (bleu) or vocabulary variety are present in the base corpus co. the pre-translation does not change anything, and even reduces these features. ## conclusion based on the assumption that proximity between languages could facilitate learning by a neural translation model, we locally pre-translated words and morphosyntactic structures in the source language. no significant results were observed. some results have deteriorated. we tried to correlate these results with quantifiable features of the corpora but no clear causal relationships appeared. several hypotheses are possible. either the pre-translation is not thorough enough and more components need to be pre-translated to see a notable positive effect. in this case, a more massive intervention should be considered, or even coordinated with an intervention on the target. or the linguistic characteristics of the two languages do not allow any progress through pre-translation. this could be confirmed by carrying out the same work on another language, for example sov language (e.g. basque) and/or with a poorer morphology svo language (english). we have observed three corpus sizes. there is a slight improvement for the corpus of 500k words (vs 50k and 2.5m). to better understand the reasons for this behaviour, we propose to repeat the experiments with intermediate corpus sizes . train.2m5 val pud ted.test c0 2,527,216 2,964 1,000 2,929