Target-Side Data Augmentation for Sequence Generation

1 INTRODUCTION . Sequence generation is an important class of machine learning tasks and has made great progress in recent years ( Vaswani et al. , 2017 ; Hassan et al. , 2018 ; Zhou et al. , 2020 ; Stiennon et al. , 2020 ) . In general , it is to generate sequences of interest based on a given source input . For example , in text summarization , the source input is a long document while the target sequence is a summary . In machine translation , the source input is a sentence in a language while the target output is a sentence in another language . Autoregressive ( AR ) methods , which generate tokens one by one autoregressively to leverage previously generated tokens , are widely used in most sequence generation problems ( Gatt & Krahmer , 2018 ; Brown et al. , 2020 ) . Data augmentation is a crucial technology for many machine learning tasks including sequence generation . The goal of data augmentation is to increase the number of training examples without incurring additional efforts of manually labeling , which could be costly or even impossible . For example , Wang et al . ( 2018 ) add random noise into input sequences , Gao et al . ( 2019 ) use a pretrained language model to synthesize pseudo input sequences , and Edunov et al . ( 2018 ) use backtranslation to synthesize pseudo-pairs for machine translation . Furthermore , we list more works about the data augmentation method in Section 2 . While these works have demonstrated promising improvements , an inherent aspect of AR sequence generation is generally overlooked . As illustrated in Figure 1 , in AR generation , each token is generated based on two kinds of input information : the original source inputs and previously generated target-side tokens ( aka target-side inputs ) . While most existing data augmentation works focus on enhancing source inputs , the augmentation of target-side inputs is under-explored . As far as we know , only there are only a few works ( Mihaylova & Martins , 2019 ; Zhang et al. , 2019 ) weakly related to this problem , which are to address the exposure bias issue ( Bengio et al. , 2015 ) of the teacher forcing training . Those works adjust the sequence in token-level by randomly sampling target-side words from ground truth and model output . However , the token level adjustment may not fully leverage the potential of target input . This work focuses on target-side data augmentation for sequence generation and proposes a sequence-level target-side augmentation method . In training , we first feed the ground truth target tokens to the decoder to generate the output probability distributions ( over the target vocabulary ) for each target position . Then these distributions are multiplied with the target-side token embeddings to build soft pseudo tokens as enhanced target-side inputs . Inspired by the success of mixup ( Zhang et al. , 2017 ) , the pseudo tokens are further mixed up with original ground truth target tokens to build synthetic target-side sequences . Next , our model is trained with both ground truth data and synthetic data . Note that we only change the target tokens for decoder input rather than target outputs or labels . Our model can benefit from the stochastic representation of target input tokens to achieve better performance . The inference process is kept the same as the conventional method . Moreover , no extra data or model parameters are required during inference . That is , we do not introduce any additional burden on model inference . We conduct experiments on various sequence generation tasks , including dialog generation , machine translation , and abstractive summarization . On five datasets , our method outperforms standard AR training without data augmentation , as well as existing data augmentation methods . Our main contributions are as follows : Problem : We shine a light on a previously neglected problem , namely , the target-side inputs argumentation ( TIA ) for autoregressive sequence generation . Method : We propose a new algorithm , the Soft Sequence-level Target-side Inputs Argumentation ( SSTIA ) , which constructs synthetic target-side input tokens by generating soft pseudo-token sequences . Experiment : We test our method on three sequence generation tasks with five data sets . Our method achieves good improvements over strong baselines , which reveals the importance of targetside data augmentation and the effectiveness of our algorithm . We want to call more attention from the community to design better methods for target-side data augmentation . 2 RELATED WORK . Data augmentation has been widely used in almost all tasks of deep learning and has been widely surveyed before ( Shorten & Khoshgoftaar , 2019 ; Wen et al. , 2020 ; Feng et al. , 2021 ) . In this section , we only briefly discuss works related to our method . Rule-based augmentation : Some works propose to use heuristic rules to augment the source data . For example , Wang et al . ( 2018 ) randomly replace words in sentences with another word in the vocabulary , Guo et al . ( 2020 ) softly combine sequences in the training set , Shen et al . ( 2020 ) randomly erase part of a sequence to build restricted views , and Sennrich et al . ( 2016b ) use pervasive dropout to enrich data . However , those methods target to enhance source input and do not pay attention to the target input . Model-based augmentation : In this category of methods , deep learning models are used to generate the augmented data . For example , Gao et al . ( 2019 ) leverage a language model to enhance the source tokens , and Kumar et al . ( 2020 ) use BERT ( Devlin et al. , 2018 ) , GPT-2 ( Radford et al. , 2019 ) , and BART ( Lewis et al. , 2019 ) to generate pseudo data . For tasks with high duality , e.g. , machine translation , back translation is widely used ( Sennrich et al. , 2016a ; Edunov et al. , 2018 ; Prabhumoye et al. , 2018 ; Hoang et al. , 2018 ) . Usually , large-scale monolingual data is used to build pseudo training data . Similar to rule-based methods , the model-based methods do not pay much effort to the target input either . Knowledge distillation : Knowledge distillation is also a widely used method for sequence generation ( Hinton et al. , 2015 ; Kim & Rush , 2016 ; He et al. , 2019 ; Gou et al. , 2021 ) . In this method , teacher models are trained to generate synthetic labels for student models . However , for student models , the target input and output of the decoder are still same . On the contrary , our methods focus on changing the target input , therefore the decoder input and output are different . And we do not have any intention of model compression . Scheduled sampling : Scheduled sampling methods are invented to solve the exposure-bias problem ( Bengio et al. , 2015 ; Mihaylova & Martins , 2019 ; Zhang et al. , 2019 ) . Although these methods also change the target input , they can only work in the token-level adaptation . In contrast , our method builds sequence-level synthetic data to improve training performance . This is an important difference between our method and scheduled sampling methods . Besides , when using sentencelevel oracle , OR-NMT ( Zhang et al. , 2019 ) need full decoding and training time could be 5x slower . Reinforcement learning : The refinement learning ( RL ) based sequence generation methods ( Ranzato et al. , 2015 ; Shen et al. , 2015 ; Norouzi et al. , 2016 ) can directly optimize the metrics that are used in test time and the target input of decoder is not always necessary . However , successfully training a RL-based sequence generation is still challenging due to its instability ( Wu et al. , 2018 ) . Therefore , we focus on AR generation in this paper . Non-autoregressive generation Non-autoregressive ( NAR ) generation has been proposed as an alternative sequence generation paradigm , where all target tokens are generated in parallel . Although model performance has been greatly improved by many recent works ( Gu et al. , 2017 ; Qian et al. , 2020 ; Ren et al. , 2020 ) , the AR generation is still the most common approach . Therefore , we choose to study the AR generation in this work and leave the application on NAR for future work . Iterative refinement : Iterative refinement is a common way to improve performance . For example , Xia et al . ( 2017 ) proposed deliberation networks , and Lee et al . ( 2018 ) proposed to use iterative refinement in the NAR generation ( Gu et al. , 2017 ) . However , these methods need multiple passes during inference time . Therefore , the iterative NAR is slower than fully NAR method ( Gu & Kong , 2020 ) . By contrast , our method only augments the data during training and has no extra inference cost . This makes our methods applicable to more scenarios where latency is critical . 3 OUR METHOD . In this section , we first introduce the background and notations . Next , we present the target-side data augmentation framework and our proposed method . Finally , we will discuss the relation with iterative data augmentation methods . Background : Let x denote the source-side input , and let y = ( y1 , y2 , · · · , yn ) denote a target sequence to generate , where n is the sequence length , and each yi belongs to a predefined vocabulary V made up of all words or sub-words . Let D = { ( xj , yj ) } Mj=1 denote the training set with size M . In the AR generation , our task is to model the distribution P ( y|x ) . Specifically , P ( y|x ) = n∏ i=1 P ( yi|x , y < i ) ( 1 ) where y < i denotes the sub-sequence before yi , and y0 is represented by a special begin-of-sentence ( BOS ) token . We can see that the probability of generating yi conditions on both x and y < i . Conventional source-side data augmentation methods can be summarized as using a function faug to generate enhanced data set . Specifically , for any ( xj , yj ) ∈ D , we generate the source-side sequence by x̃j = faug ( xj , yj ) , and append ( x̃j , yj ) to the enhanced dataset D̃ . The faug can also solely base on x or y . This augmentation method can be rule based ( e.g. , a noise function ) or model based ( e.g. , a pre-trained language model ) . It may also contain some external knowledge or heuristic ( Wang et al. , 2018 ; Kobayashi , 2018 ; Kumar et al. , 2020 ) . Based on the original dataset D and the augmented dataset D̃ , the training objective function is defined as follows : L = αLD + ( 1− α ) LD̃ = −α 1 M M∑ i=1 logP ( yi|xi ) − ( 1− α ) 1 M̃ M̃∑ i=1 logP ( yi|x̃i ) = − ( 1− α ) 1 M M∑ i=1 ni∑ j=1 logP ( yij |xi , yi < j ) − α 1 M̃ M̃∑ i=1 ni∑ j=1 logP ( yij |x̃i , yi < j ) , ( 2 ) The parameter α in Equation 2 balances the loss weight on real dataD and synthetic data D̃ , and the ni denotes the length of yi . Enhancing the target-side conditional input : Different from faug which focuses on enhancing the source-side input , we propose a new method that augments the previously generated target tokens , i.e. , the y < i in equation 1 . We create an augmented training data set D̆ = { ( xk , yk , y̆k ) } Mk=1 , where the y̆k is the augmented sequence serving as the conditional input of the target sequence . The workflow is shown in Figure 2 ( a ) . The loss function on D̆ is defined as follows : LD̆ = − 1 M M∑ i=1 ni∑ j=1 logP ( yij |xi , y̆i < j ) . ( 3 ) Some recent study ( Chen et al. , 2020 ; Shen et al. , 2020 ) have found that maximizing agreement between multiple augmented data that are generated from a single piece of data can improve model performance . Inspired by this , we design a consistency loss Lc between synthetic data and real data to stabilize the training . It is defined as LC = − 1 M M∑ i=1 ni∑ j=1 D ( P ( ·|xi , yi < j ) , P ( ·|xi , y̆i < j ) ) . ( 4 ) The D in Equation 4 can be any function to measure the divergence of two distributions , e.g. , Kullback–Leibler divergence or Jensen–Shannon divergence . We found that using Lc can stabilize the training . Our final loss function is defined as : L = ( 1− α ) LD + αLD̆ + βLC ( 5 ) Empirically , we found β = 1 is a good default value and used in all experiments . Augmented data generation : We use model-based method to generate the augmented sequence y̆ . However , we do not need external pre-trained models like ( Gao et al. , 2019 ) . Specifically , ( 1 ) We obtain the logits l̂ij for any j ∈ { 1 , 2 , · · · , ni } , which are the values before softmax , of all target tokens . Intuitively , l̂ij = dec ( enc ( x i ) , yi < j ) , where enc and dec denote the encoder and decoder of the sequence generation model and l̂ij is a |V|-dimensional vector . Note that we feed the groundtruth y < j into the decoder to obtain l̂ij . ( 2 ) We obtain the y̆ij as follows : y̆ij = ( γ exp ( l̂ij/T ) ∑|V| v=1 exp ( l̂ i j [ v ] /T ) + ( 1− γ ) yij ) WEmb , ( 6 ) where ( i ) WEmb is the target word embedding with size |V| × d ; ( ii ) T is the softmax temperature ; ( iii ) the square brackets [ · ] denotes the subscription of vector ; ( iv ) γ is the mixup ratio and we use the same value as α in Equation 5 for all experiments . The y̆ij is a mixture of two parts : ( i ) the embedding of the groundtruth yij ; ( ii ) a soft representation , which is the product between the target embedding and a distribution output by a sequence generation model . Empirically , we found T is an important hyper-parameter and more details can be found in Section 4.4 . In addition , as the modern sequence generation follows the encoder-decoder architecture , we can further reduce the augmentation cost by reusing the encoder output because it is not changed during decoding . Multi-round enhancement : Similar to iterative back-translation ( Hoang et al. , 2018 ) , our model can be easily extended to multiple rounds . That is , we can use the Equation 6 multiple times , where the logits l̂ij need to be replaced by the previous output . The experimental results suggest that too many iterations only give marginal extra gain . Therefore , our experiments mainly focus on single round augmentation .

Summary: This paper introduces a simple technique for target-side data augmentation. The high-level idea is to generate a "soft token" embedding by interpolating the target side word embeddings, with the output distribution generated by an initial model being the interpolation weight (after being adjusted by a temperature parameter $T$). The resulting training loss is in three terms: (1) original cross-entropy loss; (2) cross-entropy loss with the soft token embedding as the input; (3) a consistency loss controlling the distribution divergence of (1) and (2).  Results show that the proposed data augmentation method is very helpful for the dialog generation task, while also being somewhat helpful with neural machine translation for abstractive summarization. Ablation studies show that the weight of (1) cannot be too low, and that the optimal temperature parameter has some negative correlation with the vocabulary size.