Language Modeling, Lexical Translation, Reordering: The Training Process of NMT through the Lens of Classical SMT (2021)

Abstract: Differently from the traditional statistical MT that decomposes the translation task into distinct separately learned components, neural machine translation uses a single neural network to model the entire translation process. Despite neural machine translation being de-facto standard, it is still not clear how NMT models acquire different competences over the course of training, and how this mirrors the different models in traditional SMT. In this work, we look at the competences related to three core SMT components and find that during training, NMT first focuses on learning target-side language modeling, then improves translation quality approaching word-by-word translation, and finally learns more complicated reordering patterns. We show that this behavior holds for several models and language pairs. Additionally, we explain how such an understanding of the training process can be useful in practice and, as an example, show how it can be used to improve vanilla non-autoregressive neural machine translation by guiding teacher model selection.

## introduction in the last couple of decades, the two main machine translation paradigms have been statistical and neural mt. statistical mt (smt) decomposes the translation task into several components (e.g., lexical translation probabilities, alignment probabilities, target-side language model, etc.) which are learned separately and then combined in a translation model. differently, neural mt (nmt) models the entire translation process with a single neural network that is trained end-to-end. although joint training of all the components is one of the obvious nmt strengths, this is also one of its challenging aspects. while smt models different competences with distinct model components and, therefore, can easily validate and/or improve each of them, nmt acquires these competences within the same network over the course of training. even though previous work shows how to improve some of the competences in nmt, e.g., by using lexical translation probabilities, phrase memories, target-side lm, alignment information @xcite @xcite @xcite @xcite @xcite @xcite @xcite @xcite @xcite @xcite , it is still not clear how and when nmt acquires these competences during training. for example, are there any stages where nmt focuses on different aspects of translation, e.g., fluency (agreement on the target side) or adequacy (i.e. connection to the source), or does it improve everything at the same rate? does it learn word-by-word translation first and more complicated patterns later, or is there a different behavior? this is especially interesting in light of a recent work analyzing how nmt balances the two different types of context: the source and prefix of the target sentence @xcite . as it turns out, changes in nmt training are non-monotonic and form several distinct stages (e.g., stages changing direction from decreasing influence of source to increasing), which hints that the nmt training consists of stages with qualitatively different changes. in this paper, we try to understand what happens in these stages by analyzing translations generated at different training steps. specifically, we focus on the aspects related to the three core smt components: target-side language modeling, lexical translation, and reordering. we find that during training, nmt focuses on these aspects in the specified above order. intuitively, it starts by hallucinating frequent n-grams and sentences in the target language, then comes close to word-by-word translation, and finally learns more complicated reordering patterns. we confirm these findings for several models, lstm and transformer, and different modeling paradigms, encoder-decoder and decoder-only, i.e. lm-style machine translation where a left-to-right language model is trained on the concatenation of source and target sentences. finally, we show how such an understanding of the training process can be useful in practice. namely, we note that during a large part of training, a model's quality (e.g. bleu and token-level predictive accuracy) changes little, but reordering becomes more complicated. this means that by using different training checkpoints, we can get highquality translations of varying complexity, which is useful in settings where data complexity matters. for example, guiding teacher model selection for distillation in non-autoregressive machine translation (nat) can improve the quality of a vanilla nat model by more than 1 bleu. our contributions are as follows: ## training stages: the two viewpoints in this section, we introduce two points of view on the nmt training process. the first one comes from previous work showing distinct stages in nmt training. these stages are formed by looking at a model's internal workings and changes in the way it balances source and target information when forming a prediction. the second point of view is from this work: we take model translations at different training steps and look at some of their aspects mirroring, in a way, core smt components. while these two points of view are complete opposites (one sees only the model's innermost workings, the other -only its output), only taken together they can fully describe the training process. we start from the first, abstract, stages, then show how these inner processes look on the outside and conclude with one of the immediate practical applications of our analysis (section 6). ## transformer training stages in this section, we discuss the standard encoderdecoder transformer. in the next section, we mention differences with several other models. we first analyze the results for each of the three competences and then characterize the stages based on these practical observations. in all figures, we show the abstract stages with vertical lines to link the results to the changes in token contributions. ## other nmt models in this section, we compare different architectures within the same encoder-decoder framework (transformer vs lstm), and different frameworks with the transformer architecture (encoder-decoder vs decoder-only). overall, we find that all models follow the behavior described in section 4.4; here we discuss some of their differences. transformer vs lstm. as might be expected from the low bleu scores ( regarding the latter, it is not clear whether this is because of the lower model capacity or because lstm has an inductive bias towards more monotonic alignments; we leave this to future work. encoder-decoder vs decoder-only. ## practical implications we showed that during a large part of the training, the translation quality (e.g., bleu) changes little, but the alignments become less monotonic. intuitively, the translations become more complicated while their quality remains roughly the same. one way to directly apply our analysis is to consider tasks and settings where data properties such as regularity and/or simplicity are important. for example, in neural machine translation, higher monotonicity of artificial sources was hypothesized to be a facilitating factor for back-translation (burlot and yvon, 2018); additionally, complexity of model vocabulary (see section 3). in the appendix, we show scores for all three models. the distilled data is crucial for sequence-level distillation in non-autoregressive machine translation @xcite . such examples are not limited to machine translation: in emergent languages, languages with higher 'regularity' bring learning speed advantages for communicating neural agents @xcite . in this section, we consider non-autoregressive nmt, and leave the rest to future work. ## additional related work other work connecting neural and traditional approaches include modeling modifications, such as modeling coverage and/or fertility @xcite @xcite and several other modifications @xcite @xcite , analysis of the relation between attention and word alignments @xcite , and word alignment induction from nmt models @xcite @xcite @xcite . previous analysis of nmt learning dynamics include analyzing how the trainable parameters affect an nmt model @xcite and looking at the speed of learning specific discourse phenomena in context-aware nmt @xcite . ## conclusions we analyze how nmt acquires different competencies during training and look at the competencies related to three core smt components. we find that nmt first focuses on learning target-side language modeling, then improves translation quality approaching word-by-word translation, and finally learns more complicated reordering patterns. we show that such an understanding of the training process can be useful in settings where data complexity matters and illustrate this for non-autoregressive mt; other tasks can be considered in future work. additionally, our results can contribute to the discussion of (i) 'easy' and 'difficult' task-relevant features, including 'shortcut features', and (ii) the limitations of the bleu score. fuzzy reordering score aligns each word in σ 1 to an instance of itself in σ 2 taking the first unmatched instance of the word if there is more than one. if c is the number of chunks of contiguously aligned words and m is the number of words in the source sentence, then the fuzzy reordering score is computed as this metric assigns a score between 0 and 1, where 1 indicates that the two reorderings are identical. intuitively, c is the number of times a reader would need to jump in order to read the reordering σ 1 in the order proposed by σ 2 . a larger fuzzy reordering score indicates more monotonic alignments. kendall tau distance counts the number of pairwise disagreements between two ranking lists. the larger the distance, the more dissimilar the two lists are. kendall tau distance is also called bubble-sort distance since it is equivalent to the number of swaps that the bubble sort algorithm would take to place one list in the same order as the other list. we evaluate the normalized distance, i.e. for a list of length n it is normalized by n(n-1) 2 . the normalized score is between 0 and 1, where 0 indicates that the two reorderings are identical. differences between the scores. while the first score counts only the number of chunks of contiguously aligned words, the second one takes into account only how distant the changes are. for example, let us consider two reorderings: (2, 1, 4, 3, 6, 5) and (4, 5, 6, 1, 2, 3). while for the fuzzy reordering score the least monotonic reordering is the first (more jumps for a reader), for the kendall tau score -the second (requires more permutations to reorder). as we will see in section 4.3, results for the two scores are similar. our setting. we take sentences of at least 2 words for the fuzzy reordering score and at least 10 tokens for the kendall tau distance.