Efficiently Reusing Old Models Across Languages via Transfer Learning (2020)

Abstract: Recent progress in neural machine translation (NMT) is directed towards larger neural networks trained on an increasing amount of hardware resources. As a result, NMT models are costly to train, both financially, due to the electricity and hardware cost, and environmentally, due to the carbon footprint. It is especially true in transfer learning for its additional cost of training the “parent” model before transferring knowledge and training the desired “child” model. In this paper, we propose a simple method of re-using an already trained model for different language pairs where there is no need for modifications in model architecture. Our approach does not need a separate parent model for each investigated language pair, as it is typical in NMT transfer learning. To show the applicability of our method, we recycle a Transformer model trained by different researchers and use it to seed models for different language pairs. We achieve better translation quality and shorter convergence times than when training from random initialization.

## introduction neural machine translation (nmt), the current prevalent approach to automatic translation, is known to require large amounts of parallel training sentences and an extensive amount of training time on dedicated hardware. the total training time significantly increases, especially when training strong c 2020 the authors. this article is licensed under a creative commons 3.0 licence, no derivative works, attribution, cc-by-nd. in this paper, we propose a novel view on reusing already trained "parent" models without the need to prepare a parent model in advance or modify its training hyper-parameters. furthermore, we propose a second method based on a vocabulary transformation technique that makes even larger improvements, especially for languages using an alphabet different from the re-used parent model. our transfer learning approach leads to better performance as well as faster convergence speed of the "child" model compared to training the model from scratch. we document that our methods are not restricted only to low-resource languages, but they can be used even for high-resource ones. previous transfer learning techniques @xcite rely on a shared vocabulary between the parent and child models. as a result, these techniques separately train parent model for each different child language pair. in contrast, our approach can re-use one parent model for multiple various language pairs, thus further lowering the total training time needed. in order to document that our approach is not restricted to parent models trained by us, we re-use parent model trained by different researchers: we use the winning model of wmt 2019 for czech-english language pair @xcite . the paper is organized as follows: section 2 describes the method of direct transfer learning, including our improvement of vocabulary transformation. section 3 presents the model, training data, and our experimental setup. section 4 describes the results of our methods followed by the analysis in section 5. related work is summarized in section 6 and we conclude the discussion in section 7. ## transfer learning in this work, we present the use of transfer learning to reduce the training time and improve the performance in comparison to training from random initialization even for high-resource language pairs. transfer learning is an approach of using training data from a related task to improve the accuracy of the main task in question @xcite . one of the first transfer learning techniques in nmt was proposed by @xcite . they used wordlevel nmt and froze several model parts, especially embeddings of words that are shared between parent and child model. we build upon the work of @xcite , who simplified the transfer learning technique thanks to the use of subword units @xcite in contrast to word-level nmt transfer learning @xcite and extended the applicability to unrelated languages. their only requirement, and also the main disadvantage of the method, is that the vocabulary has to be shared and constructed for the given parent and child languages jointly, which makes the parent model usable only for the particular child language pair. this substantially increases the overall training time needed to obtain the desired nmt system for the child language pair. the method of @xcite consists of three steps: (1) construct the vocabulary from both the parent and child corpora, (2) train the parent model with the shared vocabulary until convergence, and (3) continue training on the child training data. @xcite call such approaches warm-start, where we use the child language pair to influence the parent model. in our work, we focus on the so-called cold-start scenario, where the parent model is trained without a need to know the language pair in advance. therefore we cannot make any modifications of the parent training to better handle the child language pair. the cold-start transfer learning is expected to have slightly worse performance than the warm-start approach. however, it allows reusing one parent model for multiple child language pairs, which reduces the total training time in comparison to the use of warm-start transfer learning. we present two approaches: direct transfer that ignores child-specific vocabulary altogether; and transformed vocabulary, which modifies vocabulary of the already trained parent. thus, one parent model can be used for multiple child language pairs. ## experiments in this section, we first provide the details of the nmt model used in our experiments and the examined set of language pairs. we then discuss the convergence and a stopping criterion and finally present the results of our method for recycling the nmt model as well as improvements thanks to the vocabulary transformation. ## results all reported results are calculated on the test data and evaluated with sacrebleu @xcite . the results are in baselines use the same architecture, and they are trained solely on the child training data with the use of child-specific vocabulary. we compute statistical significance with a paired bootstrap resampling @xcite . we use 1000 samples and a confidence level of 0.05. statistically significant improvements are marked by ‡. ## analysis by freezing parameters to discover which transferred parameters are the most helpful for the child model and which need to be changed the most, we follow the analysis used by @xcite : when training the child, we freeze some of the parameters. based on the internal layout of the transformer model in tensor2tensor, we divide the model into four components. (i) word embeddings (shared between encoder and decoder) map each subword unit to a dense vector representation. (ii) the encoder component includes all the six feed-forward layers converting the input sequence to the deeper representation. (iii) the decoder component consists again of six feed-forward layers preparing the choice of the next output subword unit. (iv) the multi-head attention is used throughout encoder and decoder, as self-attention layers interweaved with the feed-forward layers. we run two sets of experiments: either freeze only one out of the four components and leave the rest of the model updating or freeze everything but the examined component. we also test it on two translation directions: english-to-estonian in the left hand part of figure 2 and estonian-to-english in the right hand part. in both cases, english-czech (in the corresponding direction, i.e. with english on the correct side) serves as the parent. we discuss individual components separately, indexing the experiments 1 to 8 . similarly to @xcite in domain adaptation, we observe that parent embeddings serve well in direct transfer, freezing them has a minimal impact compared to the baseline in 1 and 5 . the frozen embeddings in transformed vocabulary ( 2 , 6 ) results in significant performance drops which can be attributed to the arbitrary assignment of embeddings to new subwords. the comparison of all but embeddings frozen in 4 and 8 (transformed vocabulary) is interesting. in 8 , the performance of the network can be recovered close to the baseline by retraining either parent source embeddings or the encoder. these two components can compensate for each other. this differs from the case with english reused in the source ( 4 ) where updating embeddings to the child language is insufficient: the decoder must be updated to produce fluent output in the new target language and even with the decoder updated, the loss compared to the baseline is quite substantial. the most important component for transfer learning is generally the component handling the new language: decoder in english-to-estonian and encoder in the reverse. with this component fixed, the performance drops the most with this component fixed ( 1 , 2 , 5 , 6 ) and among the least with this component free to update ( 3 , 4 , 7 , 8 ). this confirms that at least for examined language pair, the transformer model lends itself very well to encoder or decoder re-use. other results in figure 2 reveal that the architecture can compensate for some of the training deficiencies. freezing the encoder 1 , 2 (resp. decoder for estonian-to-english 5 , 6 ) or attention is not that critical as the frozen decoder (resp. encoder). the bad result of the encoder 3 , 4 (resp. decoder 7 , 8 ) being the only non-frozen component shows that model is not capable of providing all the needed capacity for the new language, unlike the self-attention where the loss is not that large. this behaviour correlates with our intuition that the model needs to update the most the component that handles the differing language with the parent model (in our case czech). all in all, these experiments illustrate the robustness of the transformer model that it is able to train and reasonably well utilize pre-trained weights even if they are severely crippled. ## related work this paper focuses on re-using an existing nmt model in order to improve the performance in terms of training time and translation quality without any need to modify the model or pre-trained weights. @xcite presented two model modifications for multilingual mt and showed that transfer learning could be extended to transferring from the parent to the first child, followed by the second child and then the third one. they achieved improvements with dynamically updating embeddings for the vocabulary of a target language. the use of other language pairs for improving results for the target language pair has been approached from various angles. one option is to build multilingual models @xcite , ideally so that they are capable of zero-shot, i.e. translating in a translation direction that was never part of the training data. @xcite and @xcite achieve this with a unique language tag that specifies the desired target language. the training data includes sentence pairs from multiple language pairs, and the model implicitly learns translation among many languages. in some cases, it achieves zero-shot and can translate between languages never seen together. @xcite tackled the problem by creating universal embedding space across multiple languages and training many-to-one mt system. @xcite propose multi-way multi-lingual systems. their goal is to reduce the total number of parameters needed to train multiple source and target models. in all cases, the methods are dependent on a special training schedule. the lack of parallel data in low-resource language pairs can also be tackled by unsupervised translation @xcite . the general idea is to train monolingual autoencoders for both source and target languages separately, followed by mapping both embeddings to the same space and training simultaneously two models, each translating in a different direction. in an iterative training, this pair of nmt systems is further refined, each system providing training data for the other one by back-translating monolingual data @xcite . for very closely related language pairs, transliteration can be used to generate training data from a high-resourced pair to support the low-resourced one as described in @xcite . ## conclusion in this paper, we focus on a setting where existing models are re-used without any preparation for knowledge transfer of original model ahead of its training. this is a relevant and prevailing situation in academia due to computing restrictions, and industry, where updating existing models and scaling to more language pairs is essential. we evaluate and propose methods of re-using transformer nmt models for any "child" language pair regardless of the original "parent" training languages and especially showing, that no modification is better than training from scratch. the techniques are simple, effective, and applicable to models trained by others which makes it more likely that our experimental results will be replicated in practice. we showed that despite the random assignment of subwords, the transformed vocabulary improves the performance and shortens the training time of the child model compared to training from random initialization. furthermore, we showed that this approach is not restricted to low-resource languages, and we documented that the highest improvements are (expectably) due to the shared english knowledge. moreover, we confirmed the robustness of the transformer and its ability to achieve good results in adverse conditions like very fragmented subword units or parts of the network frozen. the warm-start approach by @xcite performs slightly better than our transformed vocabulary, but it needs to be trained for a significantly longer time. this leaves room for approaches that also focus on the efficiency of the training process. we perceive our approach as a technique for increasing the performance of a model without an increase in training time. thus, re-using older models in cold-start scenario of transfer learning can be used in standard nmt training pipelines without any performance or speed losses instead of random initialization as is the common practice currently.