Target-Side Data Augmentation for Sequence Generation

1 INTRODUCTION . Sequence generation is an important class of machine learning tasks and has made great progress in recent years ( Vaswani et al. , 2017 ; Hassan et al. , 2018 ; Zhou et al. , 2020 ; Stiennon et al. , 2020 ) . In general , it is to generate sequences of interest based on a given source input . For example , in text summarization , the source input is a long document while the target sequence is a summary . In machine translation , the source input is a sentence in a language while the target output is a sentence in another language . Autoregressive ( AR ) methods , which generate tokens one by one autoregressively to leverage previously generated tokens , are widely used in most sequence generation problems ( Gatt & Krahmer , 2018 ; Brown et al. , 2020 ) . Data augmentation is a crucial technology for many machine learning tasks including sequence generation . The goal of data augmentation is to increase the number of training examples without incurring additional efforts of manually labeling , which could be costly or even impossible . For example , Wang et al . ( 2018 ) add random noise into input sequences , Gao et al . ( 2019 ) use a pretrained language model to synthesize pseudo input sequences , and Edunov et al . ( 2018 ) use backtranslation to synthesize pseudo-pairs for machine translation . Furthermore , we list more works about the data augmentation method in Section 2 . While these works have demonstrated promising improvements , an inherent aspect of AR sequence generation is generally overlooked . As illustrated in Figure 1 , in AR generation , each token is generated based on two kinds of input information : the original source inputs and previously generated target-side tokens ( aka target-side inputs ) . While most existing data augmentation works focus on enhancing source inputs , the augmentation of target-side inputs is under-explored . As far as we know , only there are only a few works ( Mihaylova & Martins , 2019 ; Zhang et al. , 2019 ) weakly related to this problem , which are to address the exposure bias issue ( Bengio et al. , 2015 ) of the teacher forcing training . Those works adjust the sequence in token-level by randomly sampling target-side words from ground truth and model output . However , the token level adjustment may not fully leverage the potential of target input . This work focuses on target-side data augmentation for sequence generation and proposes a sequence-level target-side augmentation method . In training , we first feed the ground truth target tokens to the decoder to generate the output probability distributions ( over the target vocabulary ) for each target position . Then these distributions are multiplied with the target-side token embeddings to build soft pseudo tokens as enhanced target-side inputs . Inspired by the success of mixup ( Zhang et al. , 2017 ) , the pseudo tokens are further mixed up with original ground truth target tokens to build synthetic target-side sequences . Next , our model is trained with both ground truth data and synthetic data . Note that we only change the target tokens for decoder input rather than target outputs or labels . Our model can benefit from the stochastic representation of target input tokens to achieve better performance . The inference process is kept the same as the conventional method . Moreover , no extra data or model parameters are required during inference . That is , we do not introduce any additional burden on model inference . We conduct experiments on various sequence generation tasks , including dialog generation , machine translation , and abstractive summarization . On five datasets , our method outperforms standard AR training without data augmentation , as well as existing data augmentation methods . Our main contributions are as follows : Problem : We shine a light on a previously neglected problem , namely , the target-side inputs argumentation ( TIA ) for autoregressive sequence generation . Method : We propose a new algorithm , the Soft Sequence-level Target-side Inputs Argumentation ( SSTIA ) , which constructs synthetic target-side input tokens by generating soft pseudo-token sequences . Experiment : We test our method on three sequence generation tasks with five data sets . Our method achieves good improvements over strong baselines , which reveals the importance of targetside data augmentation and the effectiveness of our algorithm . We want to call more attention from the community to design better methods for target-side data augmentation . 2 RELATED WORK . Data augmentation has been widely used in almost all tasks of deep learning and has been widely surveyed before ( Shorten & Khoshgoftaar , 2019 ; Wen et al. , 2020 ; Feng et al. , 2021 ) . In this section , we only briefly discuss works related to our method . Rule-based augmentation : Some works propose to use heuristic rules to augment the source data . For example , Wang et al . ( 2018 ) randomly replace words in sentences with another word in the vocabulary , Guo et al . ( 2020 ) softly combine sequences in the training set , Shen et al . ( 2020 ) randomly erase part of a sequence to build restricted views , and Sennrich et al . ( 2016b ) use pervasive dropout to enrich data . However , those methods target to enhance source input and do not pay attention to the target input . Model-based augmentation : In this category of methods , deep learning models are used to generate the augmented data . For example , Gao et al . ( 2019 ) leverage a language model to enhance the source tokens , and Kumar et al . ( 2020 ) use BERT ( Devlin et al. , 2018 ) , GPT-2 ( Radford et al. , 2019 ) , and BART ( Lewis et al. , 2019 ) to generate pseudo data . For tasks with high duality , e.g. , machine translation , back translation is widely used ( Sennrich et al. , 2016a ; Edunov et al. , 2018 ; Prabhumoye et al. , 2018 ; Hoang et al. , 2018 ) . Usually , large-scale monolingual data is used to build pseudo training data . Similar to rule-based methods , the model-based methods do not pay much effort to the target input either . Knowledge distillation : Knowledge distillation is also a widely used method for sequence generation ( Hinton et al. , 2015 ; Kim & Rush , 2016 ; He et al. , 2019 ; Gou et al. , 2021 ) . In this method , teacher models are trained to generate synthetic labels for student models . However , for student models , the target input and output of the decoder are still same . On the contrary , our methods focus on changing the target input , therefore the decoder input and output are different . And we do not have any intention of model compression . Scheduled sampling : Scheduled sampling methods are invented to solve the exposure-bias problem ( Bengio et al. , 2015 ; Mihaylova & Martins , 2019 ; Zhang et al. , 2019 ) . Although these methods also change the target input , they can only work in the token-level adaptation . In contrast , our method builds sequence-level synthetic data to improve training performance . This is an important difference between our method and scheduled sampling methods . Besides , when using sentencelevel oracle , OR-NMT ( Zhang et al. , 2019 ) need full decoding and training time could be 5x slower . Reinforcement learning : The refinement learning ( RL ) based sequence generation methods ( Ranzato et al. , 2015 ; Shen et al. , 2015 ; Norouzi et al. , 2016 ) can directly optimize the metrics that are used in test time and the target input of decoder is not always necessary . However , successfully training a RL-based sequence generation is still challenging due to its instability ( Wu et al. , 2018 ) . Therefore , we focus on AR generation in this paper . Non-autoregressive generation Non-autoregressive ( NAR ) generation has been proposed as an alternative sequence generation paradigm , where all target tokens are generated in parallel . Although model performance has been greatly improved by many recent works ( Gu et al. , 2017 ; Qian et al. , 2020 ; Ren et al. , 2020 ) , the AR generation is still the most common approach . Therefore , we choose to study the AR generation in this work and leave the application on NAR for future work . Iterative refinement : Iterative refinement is a common way to improve performance . For example , Xia et al . ( 2017 ) proposed deliberation networks , and Lee et al . ( 2018 ) proposed to use iterative refinement in the NAR generation ( Gu et al. , 2017 ) . However , these methods need multiple passes during inference time . Therefore , the iterative NAR is slower than fully NAR method ( Gu & Kong , 2020 ) . By contrast , our method only augments the data during training and has no extra inference cost . This makes our methods applicable to more scenarios where latency is critical . 3 OUR METHOD . In this section , we first introduce the background and notations . Next , we present the target-side data augmentation framework and our proposed method . Finally , we will discuss the relation with iterative data augmentation methods . Background : Let x denote the source-side input , and let y = ( y1 , y2 , · · · , yn ) denote a target sequence to generate , where n is the sequence length , and each yi belongs to a predefined vocabulary V made up of all words or sub-words . Let D = { ( xj , yj ) } Mj=1 denote the training set with size M . In the AR generation , our task is to model the distribution P ( y|x ) . Specifically , P ( y|x ) = n∏ i=1 P ( yi|x , y < i ) ( 1 ) where y < i denotes the sub-sequence before yi , and y0 is represented by a special begin-of-sentence ( BOS ) token . We can see that the probability of generating yi conditions on both x and y < i . Conventional source-side data augmentation methods can be summarized as using a function faug to generate enhanced data set . Specifically , for any ( xj , yj ) ∈ D , we generate the source-side sequence by x̃j = faug ( xj , yj ) , and append ( x̃j , yj ) to the enhanced dataset D̃ . The faug can also solely base on x or y . This augmentation method can be rule based ( e.g. , a noise function ) or model based ( e.g. , a pre-trained language model ) . It may also contain some external knowledge or heuristic ( Wang et al. , 2018 ; Kobayashi , 2018 ; Kumar et al. , 2020 ) . Based on the original dataset D and the augmented dataset D̃ , the training objective function is defined as follows : L = αLD + ( 1− α ) LD̃ = −α 1 M M∑ i=1 logP ( yi|xi ) − ( 1− α ) 1 M̃ M̃∑ i=1 logP ( yi|x̃i ) = − ( 1− α ) 1 M M∑ i=1 ni∑ j=1 logP ( yij |xi , yi < j ) − α 1 M̃ M̃∑ i=1 ni∑ j=1 logP ( yij |x̃i , yi < j ) , ( 2 ) The parameter α in Equation 2 balances the loss weight on real dataD and synthetic data D̃ , and the ni denotes the length of yi . Enhancing the target-side conditional input : Different from faug which focuses on enhancing the source-side input , we propose a new method that augments the previously generated target tokens , i.e. , the y < i in equation 1 . We create an augmented training data set D̆ = { ( xk , yk , y̆k ) } Mk=1 , where the y̆k is the augmented sequence serving as the conditional input of the target sequence . The workflow is shown in Figure 2 ( a ) . The loss function on D̆ is defined as follows : LD̆ = − 1 M M∑ i=1 ni∑ j=1 logP ( yij |xi , y̆i < j ) . ( 3 ) Some recent study ( Chen et al. , 2020 ; Shen et al. , 2020 ) have found that maximizing agreement between multiple augmented data that are generated from a single piece of data can improve model performance . Inspired by this , we design a consistency loss Lc between synthetic data and real data to stabilize the training . It is defined as LC = − 1 M M∑ i=1 ni∑ j=1 D ( P ( ·|xi , yi < j ) , P ( ·|xi , y̆i < j ) ) . ( 4 ) The D in Equation 4 can be any function to measure the divergence of two distributions , e.g. , Kullback–Leibler divergence or Jensen–Shannon divergence . We found that using Lc can stabilize the training . Our final loss function is defined as : L = ( 1− α ) LD + αLD̆ + βLC ( 5 ) Empirically , we found β = 1 is a good default value and used in all experiments . Augmented data generation : We use model-based method to generate the augmented sequence y̆ . However , we do not need external pre-trained models like ( Gao et al. , 2019 ) . Specifically , ( 1 ) We obtain the logits l̂ij for any j ∈ { 1 , 2 , · · · , ni } , which are the values before softmax , of all target tokens . Intuitively , l̂ij = dec ( enc ( x i ) , yi < j ) , where enc and dec denote the encoder and decoder of the sequence generation model and l̂ij is a |V|-dimensional vector . Note that we feed the groundtruth y < j into the decoder to obtain l̂ij . ( 2 ) We obtain the y̆ij as follows : y̆ij = ( γ exp ( l̂ij/T ) ∑|V| v=1 exp ( l̂ i j [ v ] /T ) + ( 1− γ ) yij ) WEmb , ( 6 ) where ( i ) WEmb is the target word embedding with size |V| × d ; ( ii ) T is the softmax temperature ; ( iii ) the square brackets [ · ] denotes the subscription of vector ; ( iv ) γ is the mixup ratio and we use the same value as α in Equation 5 for all experiments . The y̆ij is a mixture of two parts : ( i ) the embedding of the groundtruth yij ; ( ii ) a soft representation , which is the product between the target embedding and a distribution output by a sequence generation model . Empirically , we found T is an important hyper-parameter and more details can be found in Section 4.4 . In addition , as the modern sequence generation follows the encoder-decoder architecture , we can further reduce the augmentation cost by reusing the encoder output because it is not changed during decoding . Multi-round enhancement : Similar to iterative back-translation ( Hoang et al. , 2018 ) , our model can be easily extended to multiple rounds . That is , we can use the Equation 6 multiple times , where the logits l̂ij need to be replaced by the previous output . The experimental results suggest that too many iterations only give marginal extra gain . Therefore , our experiments mainly focus on single round augmentation .

Summary: This paper focuses on sequence generation tasks and proposes to perform data augmentation on the target side. During training, given an input-output pair ($x$, $y$), their model first uses teacher-forcing to get the output probability $p_y$ and then gets an augmented pair ($x$, $\tilde{y}$) by using "mixup" to mix the embeddings of $y$ and $p_y$. They also propose to encourage the consistency of predictions between ($x$, $y$) and ($x$, $\tilde{y}$) by minimizing the KL divergence of their output probabilities.  They experiment on three sequence generation tasks and demonstrate improvements over baselines.