Non-Autoregressive Models for Fast Sequence Generation (2022)

Abstract: Autoregressive (AR) models have achieved great success in various sequence generation tasks. However, AR models can only generate target sequence word-by-word due to the AR mechanism and hence suffer from slow inference. Recently, non-autoregressive (NAR) models, which generate all the tokens in parallel by removing the sequential dependencies within the target sequence, have received increasing attention in sequence generation tasks such as neural machine translation (NMT), automatic speech recognition (ASR), and text to speech (TTS). In this tutorial, we will provide a comprehensive introduction to non-autoregressive sequence generation.

## type of tutorial the type of tutorial is cutting-edge. nonautoregressive generation is a newly emerging topic, which has attracted increasing attention from researchers and achieved remarkable advancement in the past several years. this is the second tutorial on this topic in the history of acl, emnlp, naacl, eacl, coling, and aacl @xcite . ## tutorial outline part i: introduction (20 min) we anticipate that at least 90% of the tutorial will cover work by other researchers. ## diversity in the past, nar sequence generation usually involves one or two languages. recently, some researchers have found that nar models are good at multilingual translation @xcite , which may stimulate the progress of nar generation in multilingual scenarios. yang feng is a senior instructor and chenze shao is a junior instructor. ## prerequisites the attendees have to understand the basics of neural networks and the sequence-to-sequence framework, including word embeddings, encoderdecoder models, and the transformer architecture. ## reading list we recommend attendees to read the following papers before the tutorial: @xcite : the parallelizable transformer network based on attention mechanisms.