Question: What is the reason the authors think that cause the worse performance of the mix-source NMT system on the TED parallel corpus and the substantial monolingual corpus compared to the baseline NMT?
Evidence: Among low-resourced scenarios, zero-resourced translation task stands in an extreme level. A zero-resourced translation task is one of the most difficult situation when there is no parallel data between the translating language pair. To the best of our knowledge, there have been yet existed a published work about using NMT for zero-resourced translation tasks up to now. In this section, we extend our strategies using the proposed multilingual NMT approach as first attempts to this extreme situation.
We employ language-specific coding and target forcing in a strategy called bridge. Unlike the strategies used in under-resourced translation task, bridge is an entire many-to-many multilingual NMT. Simulating a zero-resourced German INLINEFORM0 French translation task given the available German-English and English-French parallel corpora, after applying language-specific coding and target forcing for each corpus, we mix those data with an English-English data as a “bridge” creating some connection between German and French. We also propose a variant of this strategy that we incorporate French-French data. And we call it universal.
We evaluate bridge and universal systems on two German INLINEFORM0 French test sets. They are compared to a direct system, which is an NMT trained on German INLINEFORM1 French data, and to a pivot system, which essentially consists of two separate NMTs trained to translate from German to English and English to French. The direct system should not exist in a real zero-resourced situation. We refer it as the perfect system for comparison purpose only. In case of the pivot system, to generate a translated text in French from a German sentence, we first translate it to English, then the output sentence is fed to the English INLINEFORM2 German NMT system to obtain the French translation. Since there are more than two languages involved in those systems, we increase the number of BPE merging operations proportionally in order to reduce the number of rare words in such systems. We do not expect our proposed systems to perform well with this primitive way of building direct translating connections since this is essentially a difficult task. We report the performance of those systems in Table TABREF23 .
Unsupprisingly, both bridge and universal systems perform worse than the pivot one. We consider two possible reasons:
Our target forcing mechanism is moderately primitive. Since the process is applied after language-specific coding, the target forcing symbol is the same for all source sentences in every languages. Thus, the forcing strength might not be enough to guide the decision of the next words. Once the very first word is translated into a word in wrong language, the following words tend to be translated into that wrong language again. Table TABREF24 shows some statistics of the translated words and sentences in wrong language.
Balancing of the training corpus. Although it is not severe as in the case of mix-source system for large monolingual data, the limited number of sentences in target language can affect the training. The difference of 1.07 BLEU points between bridge and universal might explain this assumption as we added more target data (French) in universal strategy, thus reducing the unbalance in training.
Those issues would be addressed in our following future work toward the multilingual attention-based NMT.
We mix the TED parallel corpus and the substantial monolingual corpus (EPPS+NC+CommonCrawl) and train a mix-source NMT system from those data. The first result is not encouraging when its performance is even worse than the baseline NMT which is trained on the small parallel data only. Not using the same information in the source side, as we discussed in case of mix-multi-source strategy, could explain the degrading in performance of such a system. But we believe that the magnitude and unbalancing of the corpus are the main reasons.
Answer: The magnitude and unbalancing of the corpus.