Diffusion Glancing Transformer for Parallel Sequence-to-Sequence Learning (2024)

Abstract: Previously, non-autoregressive models were widely recognized as being superior in generation efficiency but inferior in generation quality due to the challenges of modeling multiple target modalities.To enhance the multi-modality modeling ability, we propose the diffusion glancing transformer, which employs a modality diffusion process and residual glancing sampling.The modality diffusion process is a discrete process that interpolates the multi-modal distribution along the decoding steps, and the residual glancing sampling approach guides the model to continuously learn the remaining modalities across the layers. Experimental results on various machine translation and text generation benchmarks demonstrate that DIFFGLAT achieves better generation accuracy while maintaining fast decoding speed compared with both autoregressive and non-autoregressive models.

## introduction the transformer @xcite has been the most widely used architecture in sequence generation, outperforming its counterparts in (almost) all downstream tasks, such as machine translation and question answering @xcite . the typical transformer architecture adopts the autoregressive decoding strategy (ar), generating tokens in a predefined order, i.e., left to right. recently, non-autoregressive generation models (nar) attract increasing attention for their fast generation speed, which are considered to sacrifice generation quality by generating the outputs simultaneously instead of sequentially. however, we argue that nar has advantages compared to ar beyond generation efficiency for the following reasons: 1) the parallel generation enables nar to remove the inductive bias of the predefined generation order, thereby liberating the potential of applications, such as molecules or proteins that have no well-defined orders; 2) furthermore, nar can utilize bi-directional context for sequence modeling and generation, whereas ar could only exploit context from one direction. although being promising, current state-of-theart nar models still fall behind the ar counterpart in terms of generation quality. the main drawback of nar lies in the difficulty of modeling multimodal distributions @xcite , i.e., one input has multiple valid outputs. for example, a source sentence could be translated into multiple different target sentences. as the nar model decodes all the tokens in parallel, it may mix tokens from different translation targets. in contrast, the ar model can output consistency tokens more easily as every prediction is conditioned on previous ones. recent progress of nar has significantly improved the multi-modal learning ability of parallel generation, including knowledge distillation @xcite , iterative decoding @xcite , latent variable modeling @xcite and revised learning objectives @xcite @xcite . however, these nar models still hardly outperform the ar baseline consistently. in this paper, we propose diffglat, which shows that nar can outperform ar in both efficiency and accuracy, without requiring knowledge distillation from ar. generally diffglat is designed within the denoising diffusion implicit model framework @xcite . first, to reduce the difficulty of learning multimodalities for nar, diffglat defines a discrete modality diffusion process that smoothly decomposes the modality learning in the data across many diffusion transition steps. with modality decomposition, each diffusion transition only includes a scheduled number of modalities, which makes the modality learning much easier for nar. and our preliminary experiments confirm the effectiveness of modality decomposition (see section 2.2). note that each diffusion transition is learned by several adjacent nar layers, and thus diffglat can stack a sufficient number of layers to model complex multi-modal distributions. besides, we proposes a residual glancing sampling technique, which adaptively adjusts the learning difficulty of each diffusion transition in a layerwise and residual manner. experiments demonstrate that the proposed dif-fglat significantly improves the quality of nar generation on standard benchmarks. using only 1 decoding iteration, diffglat can outperform all nar baselines, including iterative ones. with 3 iterative decoding steps, diffglat beats the strong ar baseline with a moderate margin (+0.47 bleu). comparisons between diffglat and ar transformer on 10 standard machine translation benchmarks, with both transformer base and big settings, consistently verify the effectiveness of diffglat. additionally, we also find that dif-fglat can slightly outperform the ar baseline on image captioning and paraphrasing tasks. these results show the extensive potential of diffglat in other generation tasks. ## preliminary given the input sequence @xmath0, x 2 , ..., x m and the target sequence @xmath1, y 2 , ..., y n , the nar model factorizes the joint probability p (y|x) with the conditional independence assumption: by conditioning on preceding tokens @xmath2, the autoregressive factorization divides the modality learning of the sequence into multiple steps where each step learns to predict one token. similar to the autoregressive factorization, the denoising diffusion models also smooth the source-target transformation by interpolating intermediate distributions between the source and the target. inspired by the learning decomposition in ar and diffusion models, we employ training procedures as in diffusion models to decompose the complex modality learning of nar into several easier diffusion transition steps. since the process of adding gaussian noise is not designed for tackling the multi-modality problem, we also explore new diffusion processes to address the problem. a preliminary study demonstrates that a learning process with a gradually growing number of modalities is beneficial for learning modalities in nar. ## the proposed diffusion-glat in this section, we will introduce diffglat, which enables parallel sequence-to-sequence learning in a denoising diffusion implicit model (ddim, @xcite framework. the primary goals of designing diffglat are: a) decomposing the modality learning to reduce the training difficulty of nar and b) achieving high generation quality with few denoising transitions to keep fast decoding. in denoising diffusion models, the parameters θ are trained to fit the data distribution q(y 0 ) by maximizing a variational lower bound @xcite : to achieve the two goals described in the beginning, we define a discrete diffusion process q mdp that adds the modalities in the data gradually as the diffusion step t decreases. additionally, the training process is equipped with the residual glancing sampling techniques p rgs θ , which further boosts the modality learning ability. therefore, we can rewrite the training objective in eq. 5 as: the modality diffusion process q mdp gradually interpolates modalities among intermediate layers, which adaptively schedules sequence modalities across the nar transformer layers. and the residual glancing sampling (rgs) samples the target tokens that are not correctly captured across neural layers to enhance the modality learning process. ## experiments different from most of the previous work for nar, we directly train our models on raw data without using data distilled from ar. to verify the effectiveness of our method, we compare diffglat with several strong nar and ar baselines on several sequence generation tasks. furthermore, analysis and ablation studies study are also conducted to demonstrate the effects of each component. ## related work the work for non-autoregressive neural machine translation can be divided into non-iterative and iterative methods. previous non-iterative methods attempt to enhance the generation quality by earning from autoregressive models @xcite , incorporating light sequential decoding @xcite , and introducing latent variables @xcite @xcite . besides, models with iterative decoding have also been developed @xcite @xcite @xcite @xcite . diffglat also has connections with diffusion models for discrete data but differs from them in both probabilistic modelling and design motivation. more discussions can be found in appendix i. ## conclusion in this work, we propose diffglat, a parallel sequence generation model trained with the modality diffusion process and residual glancing sampling. by smoothing the learning of modalities in the diffusion model framework, diffglat greatly improves the generation quality of parallel generation. compared with the autoregressive transformer, diffglat achieves superior performance in both accuracy and efficiency for multiple se-quence generation tasks, demonstrating the potential of the parallel generation paradigm.