Sequence-to-Lattice Models for Fast Translation (2021)

Abstract: Non-autoregressive machine translation (NAT) approaches enable fast generation by utilizing parallelizable generative processes. The remaining bottleneck in these models is their decoder layers; unfortunately unlike in autoregressive models (Kasai et al., 2020), removing decoder layers from NAT models significantly degrades accuracy. This work proposes a sequence-to-lattice model that replaces the decoder with a search lattice. Our approach first constructs a candidate lattice using efficient lookup operations, generates lattice scores from a deep encoder, and finally finds the best path using dynamic programming. Experiments on three machine translation datasets show that our method is faster than past non-autoregressive generation approaches, and more accurate than naively reducing the number of decoder layers.

## introduction non-autoregressive (nat) machine translation @xcite provides multi-fold speedups compared to sequential generation by parallelizing computation across positions. nat models are often compared to autoregressive models with deep architectures. however, autoregressive models themselves also admit structural changes that can give large speedups, for instance using shallow decoders @xcite or pruning the per sentence vocabulary @xcite . unfortunately porting these structural changes to nat models significantly hurts accuracy, reducing their benefits. this work proposes a sequence-to-lattice formulation for translation that yields the benefits of non-autoregressive translation while reducing the practical costs of deep decoder models. the method first constructs a candidate target lattice based on the source sentence using a variant of ibm model 2 @xcite . then each lattice edge is scored based on an encoder head. fi-nally, exact inference is performed with the viterbi algorithm @xcite . experiments on machine translation benchmarks show that this simple approach achieves fast translation without requiring an ensemble of different approaches. our code, models, and logs are available at https://github.com/ harvardnlp/cascaded-generation . many prior works have considered the problem of reducing the vocabulary for efficient machine translation @xcite @xcite . the most common approach is based on the intuition that each source word can only be translated into a small set of target words, such that taking their union gives us a reduced vocabulary. in order to find which words each source word translates into, simple statistical models @xcite are usually used to get alignments. in this work, we also use a variant of ibm model 2 to reduce the size of target-side vocabularies, but instead of finding a reduced target vocabulary per sentence, our method finds a reduced target vocabulary per position. ## approach our method formulates translation as a first-order conditional markov model. given a source sentence @xmath0, • • • , x s , the goal of translation is to produce the best target sentence @xmath1, • • • , y t under: the full set of translations can be compactly represented as a lattice where each edge score corresponds to p θ (y t |y t-1 , t, x). once a lattice is constructed and scored, the best translation can be computed using the viterbi algorithm @xcite . however, the lattice is quadratic in the vocabulary size. in order to make this approach efficient, we need to specify how to: a) generate a tractable lattice, b) score the edges, and c) parallelize the process. ## conclusion in this work, we find that using a sequence-tolattice formulation enables using much smaller model architectures for fast machine translation. our approach first generates a candidate lattice using a statistical model, then uses a transformer with a position-wise head layer to score the lattice, and finally uses the viterbi algorithm to find the best hypothesis. experiments on three machine translation benchmarks show that our simple approach is very fast yet achieves a decent accuracy.