Can Latent Alignments Improve Autoregressive Machine Translation? (2021)

Abstract: Latent alignment objectives such as CTC and AXE significantly improve non-autoregressive machine translation models. Can they improve autoregressive models as well? We explore the possibility of training autoregressive machine translation models with latent alignment objectives, and observe that, in practice, this approach results in degenerate models. We provide a theoretical explanation for these empirical results, and prove that latent alignment objectives are incompatible with teacher forcing.

## introduction latent alignment objectives, such as ctc @xcite and axe @xcite , have been recently proposed for training nonautoregressive models for machine translation @xcite . these objectives use a dynamic program to comb the space of monotonic alignments between the "gold" target sequence and the token probabilities the model predicts, thus reducing the loss from positional misalignments and focusing on the original prediction error instead. for example, consider the target sequence "there is a tiny difference between pink and magenta"; if the model's distribution favors the paraphrase "there is a very small difference between pink and magenta", substituting one token ("tiny") with two ("very small") will cause a misalignment, and result in a disproportionately large cross entropy loss. a latent alignment loss would match the predictions of both "very" and "small" with the target "tiny", while aligning the rest of the sentence properly and computing a much lower loss that focuses on this particular discrepancy. could latent alignments also benefit autoregressive models? we apply ctc and axe to standard autoregressive machine translation models. we observe that, when trained with teacher forcing @xcite , ctc reduces to the vanilla cross entropy loss because ctc assumes that the prediction sequence is longer than the target and has only one valid alignment when they are equal. we further examine axe, which does not share this assumption, and find that it yields a degenerate model that almost perfectly fits the training set but completely fails at inference time. our analysis reveals that latent alignments and teacher forcing are fundamentally incompatible. we observe that there exists a valid alignment in which the prediction p i is aligned with the target y i-1 for almost every token. simultaneously, teacher forcing feeds the model with y i-1 when computing the prediction p i , encouraging the model to simply predict its input under this alignment. while axe allows this alignment for equal-length prediction and target sequences, the phenomenon also occurs (theoretically) in ctc if the predictions are longer, and in fact, occurs in any latent alignment objective that can align a prediction p j with a target y i where @xmath0. ## background: latent alignments a latent alignment objective measures the compatibility between the target sequence y and the sequence of predicted token probabilities p by considering a subspace of possible mappings between y and p . latent alignments are typically used in non-autoregressive models for automatic speech recognition, and optical character recognition @xcite , and have recently been introduced to the task of machine translation @xcite @xcite . we describe two such objectives, beginning with an overview of the common notation and framework. monotonic alignments let @xmath1, . . . , y n be the target sequence of n tokens, and @xmath2. this is the default alignment, advancing along a's diagonal. clone target assuming the target token yi was predicted with the previous distribution pj-1, repredict yi using pj. clone prediction assuming the previous target token yi-1 was predicted with the distribution pj, reuse pj to predict the next target token yi. delimiter use the distribution pj to predict the blank token ε instead of the target token yi. this operation is akin to inserting ε into the target sequence at the i-th position. objective given an alignment α, the objective is defined as follows: since α is not provided a priori, it is necessary to aggregate over all the possible alignments (hence latent alignments), by either summation (equation 2) or maximization (equation 3 ): in practice, the negative log loss is minimized during training: dynamic programming aggregation can be done efficiently with dynamic programming, using derivations of the forward-backward algorithm (for summation, as in ctc) or the viterbi algorithm (for maximization, as in axe). these algorithms create an aggregation matrix @xmath3, where each cell represents the desired aggregation score f (sum or max) over prefixes of the target and prediction probability sequences: the dynamic program combs through the space of alignments by implicitly constructing every possibility using the set of local operators defined in ## combining ctc with teacher forcing defaults to the trivial alignment in an autoregressive setting, it is standard practice to use teacher forcing @xcite ; i.e., when predicting the i-th token, the model takes the prefix of the (gold) target sequence @xmath4. this dictates that the number of predictions is identical to the number of target tokens however, ctc assumes that the prediction sequence p is typically much longer than the target sequence y (m n), and can only inflate y via clone target and delimiter (see section 2). this leaves only one valid alignment when @xmath5: the trivial alignment α(i) = {i}. ctc will thus default to the same objective as the standard cross entropy loss. unlike ctc, the axe objective aggregates over multiple alignments even when @xmath6, because it uses both the delimiter operator (which inflates y ) as well as the clone prediction operator (which inflates p ). ## applying axe to autoregressive nmt to apply axe to autoregressive machine translation, we use a standard sequence-to-sequence transformer model @xcite trained with teacher forcing, replace the simple cross entropy loss function with axe, and add the empty token ε to the vocabulary. we remove the ε tokens after decoding. experiment setup we use fairseq @xcite to train a transformer encoder-decoder @xcite on the iwslt'14 de-en dataset @xcite . the dataset is preprocessed and tokenized into subwords with bpe @xcite using the scripts provided by fairseq. we also use the implementation's default hyperparameters: 6 layers of encoder/decoder, 512 model dimensions, 1024 hidden dimensions, 4 attention heads. we optimize with adam (kingma and ba, 2015) for 50k steps with early stopping using 4096 tokens per batch. we decode with beam search (@xmath7. ## analysis to understand how the model learns to game the axe objective, we analyze the optimal alignments chosen by the objective, and find that they allow the model to condition on the target token when trying to predict it. we prove that this is the optimal solution when combining teacher forcing and axe, and that it holds for any latent alignment objective that allows the model to align future target tokens with the current prediction. figure 3 : an example of the constant alignment that axe chooses after training the model. given the german source "danke fürs zuhören", the model tries to predict "thank you for listening". because the model is trained with teacher forcing, it can simply learn to predict its input at each position, and assume that axe will align the prediction with the previous token (which is identical to the input). for example, p 2 predicts "thank" with very high probability because teacher forcing uses the previous target y 1 as the decoder's input in the second position. notice how the final prediction p 6 is used twice to predict both "." and eos. axe finds a constant alignment we examine the alignments chosen by axe's dynamic program for a sample of training examples, and observe that they all belong to a consistent pattern: delimiter, align, align, ..., clone prediction. in other words, the chosen path skips the first prediction by emitting the blank token ε and then aligns each prediction p i with the previous target token y i-1 . the alignment synchronizes the positions at the end of the sequence by cloning the last prediction to compensate for the offset produced by the initial delimiter operator. ## conclusion this work elaborates why latent alignment objectives are incompatible with autoregressive models trained with teacher forcing. that said, teacher forcing might not be the best way to train a machine translation model @xcite @xcite , and perhaps a future alternative could reopen the discussion on applying latent alignment objectives to autoregressive models.