Machine-oriented NMT Adaptation for Zero-shot NLP tasks: Comparing the Usefulness of Close and Distant Languages (2020)

Abstract: Neural Machine Translation (NMT) models are typically trained by considering humans as end-users and maximizing human-oriented objectives. However, in some scenarios, their output is consumed by automatic NLP components rather than by humans. In these scenarios, translations’ quality is measured in terms of their “fitness for purpose” (i.e. maximizing performance of external NLP tools) rather than in terms of standard human fluency/adequacy criteria. Recently, reinforcement learning techniques exploiting the feedback from downstream NLP tools have been proposed for “machine-oriented” NMT adaptation. In this work, we tackle the problem in a multilingual setting where a single NMT model translates from multiple languages for downstream automatic processing in the target language. Knowledge sharing across close and distant languages allows to apply our machine-oriented approach in the zero-shot setting where no labeled data for the test language is seen at training time. Moreover, we incorporate multi-lingual BERT in the source side of our NMT system to benefit from the knowledge embedded in this model. Our experiments show coherent performance gains, for different language directions over both i) “generic” NMT models (trained for human consumption), and ii) fine-tuned multilingual BERT. This gain for zero-shot language directions (e.g. Spanish–English) is higher when the models are fine-tuned on a closely-related source language (Italian) than a distant one (German).

## introduction with the rapid growth of cloud computing, there are plenty of online services for a variety of natural language processing (nlp) tasks such as document classification, sentiment analysis, and spam detection. however, building them from scratch typically requires a massive amount of labeled data, which is not always publicly available and, for many tasks, is limited to high-resource languages like english. a possible solution to leverage these services in low-resource settings is using neural machine translation (nmt) in the so-called "translation-based" approach, where a text in the low-resource language is first translated into a high-resource one for which dedicated nlp tools exist. then, the translated text is processed by these downstream tools and, finally, the results are propagated back to the source language. although the translation-based approach shows promising results in low-resource settings @xcite , it still has drawbacks. first, the output quality of current nmt models is not perfect yet @xcite . second, even a good translation can alter some traits in the text, which are essential for the downstream nlp tool. this, for instance, is typical for sentiment traits, whose loss can result in final performance drops in sentiment classification tasks @xcite . finally, state-of-the-art nmt models are trained considering humans as end-users and hence optimized to maximize human-oriented objectives like fluency and semantic equivalence of the translation with respect to the source sentence. however, these objectives are not necessarily the optimal ones to exploit an nlp tool at its best. machines, in fact, are still worse than humans in handling ambiguous or overly complex sentences. this observation calls for strategies that are alternative to the human-oriented enhancement of nmt. rather, models should be adapted in a machine-oriented way that is optimal for automatic processing of their output. traditionally, nmt models are trained using parallel corpora, consisting of sentences in the source language and their human translations in the target language. recently, @xcite proposed machine-oriented reinforce (mo-reinforce), a method based on reinforcement learning to pursue machine-oriented objectives for sentence-level classification tasks. in a nutshell: given the output of the downstream classifier (i.e. a probability distribution over the labels), mo-reinforce considers the probability given to the true class as the collected reward from the downstream classifier. by maximizing the expected value of the collected reward, mo-reinforce adapts the nmt model's behavior to generate outputs that are easier to label by the downstream classifier. although nmt models adapted with mo-reinforce show promising results compared to the "generic" ones trained by only pursuing human-oriented objectives, they still need a small amount of labeled data in the source language to compute the reward that may not be available for some languages. to address this problem, we exploit multilingual nmt models @xcite , the mo-reinforce algorithm and a small quantity of data in closely-related languages. starting from a multilingual nmt system used to translate texts from n low-resource languages into a resource-rich one, the mo-reinforce algorithm is run in three different conditions by using source data in: i) the same language of the test set (tuning on italian -testing on italian); ii) a different language, but closely related to the one of the test set, (italian -spanish), and iii) a different and distant language (german -spanish). the main goal of these experiments is to show that mo-reinforce leveraging a multilingual nmt and data in a closely-related language is able to overcome the lack of source labelled data in a specific task. moreover, recently, multilingual bert @xcite has shown good performance when finetuned for downstream tasks. multilingual bert is a pre-trained model built on the union of unlabeled data for more than 100 languages. the availability of unlabeled text in significant quantities in different languages helps this model to extract valuable knowledge about the languages resulting in good performance for different tasks. to strengthen the capability of the nmt system to represent the source sentence, we try different approaches to incorporate multilingual bert in the nmt system's encoder. our goal, in this case, is to show that bert-based nmt systems can benefit from the knowledge embedded in bert, particularly in zero-shot language directions. we focus on a 4-class sentence classification task (news classification), which is harder compared to the binary task (polarity detection) chosen in @xcite . we evaluate the translation-based approach from german, italian, and spanish (simulating low-resource language settings) into english. it is important to remark that, although our source languages are not low-resource ones, their choice is motivated by the availability of standard benchmark for a comparative evaluation in a simulated lowresource scenario. the results show that: pre-training a neural network or parts of it with existing models is a common approach in several nlp tasks and it allows developers to speed up the training, to leverage different types of training data and to improve the overall performance of the learning system. among various solutions, word2vec @xcite and its variants @xcite have been the first resources used to pre-training the embeddings in an nmt system. they provide embedded vectors of individual words and have been widely used in nlp. recently, pre-trained language models (lm) showed better performance when fine-tuned for downstream tasks. elmo @xcite is among the first pre-trained lms, which is based on bi-lstm architecture trained on monolingual data. the authors showed that combining the representations from different layers obtains contextual-aware word representations that can be used for other nlp tasks. right after elmo, bert @xcite was proposed based on the encoder of transformer @xcite . this model was trained on unlabeled data using two loss functions: i) masked language model (mlm) and ii) next sentence prediction (nps). this pre-trained model showed outstanding performance when fine-tuned for a variety of nlp tasks. there are many variants of bert proposed after, among them: @xcite lample (2019 specifically for nmt, different attempts have been done to integrate pre-trained lms in the sequenceto-sequence model. among others, elmo was used for initializing the embedding layer in the nmt system @xcite . @xcite used bert for initializing the encoder or embedding layer of the nmt systems. then the bert models are fixed or fine-tuned along with other variables of the model. in @xcite , a method was proposed to fuse the representations obtained from bert with each layer of the encoder and decoder in the nmt model through attention mechanisms. we take a similar approach to @xcite in initializing source embedding or the encoder of the nmt system while training a generic nmt systems. here, this is done for the first time in a machine-oriented setting and in zero-shot conditions. ## conclusion in this paper, we proposed a multilingual extension of the mo-reinforce algorithm able to work in zeroshot settings. our solution takes advantage of a multilingual nmt model, which translates texts from different low-resource languages into english. to mitigate the lack of data in zero-shot languages, we also incorporated the multilingual bert with different approaches in the nmt model. our evaluation shows that using generic nmt systems in the translation-based approach works better than the multilingual bert in zero-shot settings. furthermore, the shared knowledge between the source languages allows mo-reinforce to leverage the labeled data in one language to adapt the nmt model to pursue machine-oriented objectives in other languages, even in zero-shot settings. our results show that data in closely-related languages can help to cope with the lack of task-specific resources and confirm the capability of mo-reinforce to leverage and transfer information across languages. the best result in zero-shot settings is obtained with the nmt system by incorporating multilingual bert as embeddings adapted on closely-related language. however, for the language with a small amount of data, the best approach uses the multilingual bert as the embedding for the source side of the nmt system by enabling the dropout while generating the translation candidates. our future works will consider fine-tuning the multilingual bert variables along with other variables of the model in nmt systems, which can obtain a language-agnostic representation in the bert model and be helpful in zero-shot settings.