Year: 2021

Abstract:
Autoregressive sequence generation, a prevalent task in machine learning and natural language processing, generates every target token conditioned on both a source input and previously generated target tokens. Previous data augmentation methods, which have been shown to be effective for the task, mainly enhance source inputs (e.g., injecting noise into the source sequence by random swapping or masking, back translation, etc.) while overlooking the target-side augmentation. In this work, we propose a target-side augmentation method for sequence generation. In training, we use the decoder output probability distributions as soft indicators, which are multiplied with target token embeddings, to build pseudo tokens. These soft pseudo tokens are then used as target tokens to enhance the training. We conduct comprehensive experiments on various sequence generation tasks, including dialog generation, machine translation, and abstractive summarization. Without using any extra labeled data or introducing additional model parameters, our method significantly outperforms strong baselines. The code is available at https://github.com/TARGET-SIDE-DATA-AUG/ TSDASG.

heading: 1 INTRODUCTION
text: Sequence generation is an important class of machine learning tasks and has made great progress in recent years (Vaswani et al., 2017; Hassan et al., 2018; Zhou et al., 2020; Stiennon et al., 2020). In general, it is to generate sequences of interest based on a given source input. For example, in text summarization, the source input is a long document while the target sequence is a summary. In machine translation, the source input is a sentence in a language while the target output is a sentence in another language. Autoregressive (AR) methods, which generate tokens one by one autoregressively to leverage previously generated tokens, are widely used in most sequence generation problems (Gatt & Krahmer, 2018; Brown et al., 2020).
Data augmentation is a crucial technology for many machine learning tasks including sequence generation. The goal of data augmentation is to increase the number of training examples without incurring additional efforts of manually labeling, which could be costly or even impossible. For example, Wang et al. (2018) add random noise into input sequences, Gao et al. (2019) use a pretrained language model to synthesize pseudo input sequences, and Edunov et al. (2018) use backtranslation to synthesize pseudo-pairs for machine translation. Furthermore, we list more works about the data augmentation method in Section 2.
While these works have demonstrated promising improvements, an inherent aspect of AR sequence generation is generally overlooked. As illustrated in Figure 1, in AR generation, each token is generated based on two kinds of input information: the original source inputs and previously generated target-side tokens (aka target-side inputs). While most existing data augmentation works focus on enhancing source inputs, the augmentation of target-side inputs is under-explored. As far as we know, only there are only a few works (Mihaylova & Martins, 2019; Zhang et al., 2019) weakly related to this problem, which are to address the exposure bias issue (Bengio et al., 2015) of the teacher forcing training. Those works adjust the sequence in token-level by randomly sampling target-side words from ground truth and model output. However, the token level adjustment may not fully leverage the potential of target input.
This work focuses on target-side data augmentation for sequence generation and proposes a sequence-level target-side augmentation method. In training, we first feed the ground truth target tokens to the decoder to generate the output probability distributions (over the target vocabulary) for each target position. Then these distributions are multiplied with the target-side token embeddings to build soft pseudo tokens as enhanced target-side inputs. Inspired by the success of mixup (Zhang et al., 2017), the pseudo tokens are further mixed up with original ground truth target tokens to build synthetic target-side sequences. Next, our
model is trained with both ground truth data and synthetic data. Note that we only change the target tokens for decoder input rather than target outputs or labels. Our model can benefit from the stochastic representation of target input tokens to achieve better performance. The inference process is kept the same as the conventional method. Moreover, no extra data or model parameters are required during inference. That is, we do not introduce any additional burden on model inference.
We conduct experiments on various sequence generation tasks, including dialog generation, machine translation, and abstractive summarization. On five datasets, our method outperforms standard AR training without data augmentation, as well as existing data augmentation methods.
Our main contributions are as follows:
Problem: We shine a light on a previously neglected problem, namely, the target-side inputs argumentation (TIA) for autoregressive sequence generation.
Method: We propose a new algorithm, the Soft Sequence-level Target-side Inputs Argumentation (SSTIA), which constructs synthetic target-side input tokens by generating soft pseudo-token sequences.
Experiment: We test our method on three sequence generation tasks with five data sets. Our method achieves good improvements over strong baselines, which reveals the importance of targetside data augmentation and the effectiveness of our algorithm. We want to call more attention from the community to design better methods for target-side data augmentation.

heading: 2 RELATED WORK
text: Data augmentation has been widely used in almost all tasks of deep learning and has been widely surveyed before (Shorten & Khoshgoftaar, 2019; Wen et al., 2020; Feng et al., 2021). In this section, we only briefly discuss works related to our method.
Rule-based augmentation: Some works propose to use heuristic rules to augment the source data. For example, Wang et al. (2018) randomly replace words in sentences with another word in the vocabulary, Guo et al. (2020) softly combine sequences in the training set, Shen et al. (2020) randomly erase part of a sequence to build restricted views, and Sennrich et al. (2016b) use pervasive dropout to enrich data. However, those methods target to enhance source input and do not pay attention to the target input.
Model-based augmentation: In this category of methods, deep learning models are used to generate the augmented data. For example, Gao et al. (2019) leverage a language model to enhance the source tokens, and Kumar et al. (2020) use BERT (Devlin et al., 2018), GPT-2 (Radford et al., 2019), and BART (Lewis et al., 2019) to generate pseudo data. For tasks with high duality, e.g., machine translation, back translation is widely used (Sennrich et al., 2016a; Edunov et al., 2018; Prabhumoye et al., 2018; Hoang et al., 2018). Usually, large-scale monolingual data is used to build pseudo training data. Similar to rule-based methods, the model-based methods do not pay much effort to the target input either.
Knowledge distillation: Knowledge distillation is also a widely used method for sequence generation (Hinton et al., 2015; Kim & Rush, 2016; He et al., 2019; Gou et al., 2021). In this method, teacher models are trained to generate synthetic labels for student models. However, for student models, the target input and output of the decoder are still same. On the contrary, our methods focus on changing the target input, therefore the decoder input and output are different. And we do not have any intention of model compression.
Scheduled sampling: Scheduled sampling methods are invented to solve the exposure-bias problem (Bengio et al., 2015; Mihaylova & Martins, 2019; Zhang et al., 2019). Although these methods also change the target input, they can only work in the token-level adaptation. In contrast, our method builds sequence-level synthetic data to improve training performance. This is an important difference between our method and scheduled sampling methods. Besides, when using sentencelevel oracle, OR-NMT (Zhang et al., 2019) need full decoding and training time could be 5x slower.
Reinforcement learning: The refinement learning (RL) based sequence generation methods (Ranzato et al., 2015; Shen et al., 2015; Norouzi et al., 2016) can directly optimize the metrics that are used in test time and the target input of decoder is not always necessary. However, successfully training a RL-based sequence generation is still challenging due to its instability (Wu et al., 2018). Therefore, we focus on AR generation in this paper.
Non-autoregressive generation Non-autoregressive (NAR) generation has been proposed as an alternative sequence generation paradigm, where all target tokens are generated in parallel. Although model performance has been greatly improved by many recent works (Gu et al., 2017; Qian et al., 2020; Ren et al., 2020), the AR generation is still the most common approach. Therefore, we choose to study the AR generation in this work and leave the application on NAR for future work.
Iterative refinement: Iterative refinement is a common way to improve performance. For example, Xia et al. (2017) proposed deliberation networks, and Lee et al. (2018) proposed to use iterative refinement in the NAR generation (Gu et al., 2017). However, these methods need multiple passes during inference time. Therefore, the iterative NAR is slower than fully NAR method (Gu & Kong, 2020). By contrast, our method only augments the data during training and has no extra inference cost. This makes our methods applicable to more scenarios where latency is critical.

heading: 3 OUR METHOD
text: In this section, we first introduce the background and notations. Next, we present the target-side data augmentation framework and our proposed method. Finally, we will discuss the relation with iterative data augmentation methods.
Background: Let x denote the source-side input, and let y = (y1, y2, · · · , yn) denote a target sequence to generate, where n is the sequence length, and each yi belongs to a predefined vocabulary V made up of all words or sub-words. Let D = {(xj , yj)}Mj=1 denote the training set with size M . In the AR generation, our task is to model the distribution P (y|x). Specifically,
P (y|x) = n∏
i=1
P (yi|x, y<i) (1)
where y<i denotes the sub-sequence before yi, and y0 is represented by a special begin-of-sentence (BOS) token. We can see that the probability of generating yi conditions on both x and y<i.
Conventional source-side data augmentation methods can be summarized as using a function faug to generate enhanced data set. Specifically, for any (xj , yj) ∈ D, we generate the source-side sequence by x̃j = faug(xj , yj), and append (x̃j , yj) to the enhanced dataset D̃. The faug can also solely base on x or y. This augmentation method can be rule based (e.g., a noise function) or model based (e.g., a pre-trained language model). It may also contain some external knowledge or heuristic (Wang et al., 2018; Kobayashi, 2018; Kumar et al., 2020).
Based on the original dataset D and the augmented dataset D̃, the training objective function is defined as follows:
L = αLD + (1− α)LD̃
= −α 1 M M∑ i=1 logP (yi|xi)− (1− α) 1 M̃ M̃∑ i=1 logP (yi|x̃i)
= −(1− α) 1 M M∑ i=1 ni∑ j=1 logP (yij |xi, yi<j)− α 1 M̃ M̃∑ i=1 ni∑ j=1 logP (yij |x̃i, yi<j),
(2)
The parameter α in Equation 2 balances the loss weight on real dataD and synthetic data D̃, and the ni denotes the length of yi.
Enhancing the target-side conditional input: Different from faug which focuses on enhancing the source-side input, we propose a new method that augments the previously generated target tokens, i.e., the y<i in equation 1. We create an augmented training data set D̆ = {(xk, yk, y̆k)}Mk=1, where the y̆k is the augmented sequence serving as the conditional input of the target sequence. The workflow is shown in Figure 2(a).
The loss function on D̆ is defined as follows:
LD̆ = − 1
M M∑ i=1 ni∑ j=1 logP (yij |xi, y̆i<j). (3)
Some recent study (Chen et al., 2020; Shen et al., 2020) have found that maximizing agreement between multiple augmented data that are generated from a single piece of data can improve model performance. Inspired by this, we design a consistency loss Lc between synthetic data and real data to stabilize the training. It is defined as
LC = − 1
M M∑ i=1 ni∑ j=1 D(P (·|xi, yi<j), P (·|xi, y̆i<j)). (4)
The D in Equation 4 can be any function to measure the divergence of two distributions, e.g., Kullback–Leibler divergence or Jensen–Shannon divergence. We found that using Lc can stabilize the training.
Our final loss function is defined as:
L = (1− α)LD + αLD̆ + βLC (5)
Empirically, we found β = 1 is a good default value and used in all experiments.
Augmented data generation: We use model-based method to generate the augmented sequence y̆. However, we do not need external pre-trained models like (Gao et al., 2019). Specifically,
(1) We obtain the logits l̂ij for any j ∈ {1, 2, · · · , ni}, which are the values before softmax, of all target tokens. Intuitively, l̂ij = dec(enc(x
i), yi<j), where enc and dec denote the encoder and decoder of the sequence generation model and l̂ij is a |V|-dimensional vector. Note that we feed the groundtruth y<j into the decoder to obtain l̂ij .
(2) We obtain the y̆ij as follows:
y̆ij =
( γ
exp(l̂ij/T )∑|V| v=1 exp(l̂ i j [v]/T ) + (1− γ)yij
) WEmb, (6)
where (i) WEmb is the target word embedding with size |V| × d; (ii) T is the softmax temperature; (iii) the square brackets [·] denotes the subscription of vector; (iv) γ is the mixup ratio and we use the same value as α in Equation 5 for all experiments. The y̆ij is a mixture of two parts: (i) the embedding of the groundtruth yij ; (ii) a soft representation, which is the product between the target embedding and a distribution output by a sequence generation model. Empirically, we found T is an important hyper-parameter and more details can be found in Section 4.4. In addition, as the modern sequence generation follows the encoder-decoder architecture , we can further reduce the augmentation cost by reusing the encoder output because it is not changed during decoding.
Multi-round enhancement: Similar to iterative back-translation (Hoang et al., 2018), our model can be easily extended to multiple rounds. That is, we can use the Equation 6 multiple times, where the logits l̂ij need to be replaced by the previous output. The experimental results suggest that too many iterations only give marginal extra gain. Therefore, our experiments mainly focus on single round augmentation.

heading: 4 EXPERIMENT
text: We conduct experiments on various sequence generation tasks. The experiment details and results are in the following sections. Our implementation is based on the FairSeq framework (Ott et al., 2019). We use Transformer (Vaswani et al., 2017) network architecture in all experiments with different model sizes, which are adjusted according to the data size. During training, we use Adam optimizer (Kingma & Ba, 2014) with Adam β = (0.9, 0.98) and invert sqrt learning rate scheduler. Meanwhile, we used label smoothing of value 0.1, which is the same value as Vaswani et al. (2017).

heading: 4.1 DIALOG GENERATION
text: We conduct experiments on two commonly used dialog generation data sets: the DailyDialog (Li et al., 2017) for single-turn dialog generation and Persona-Chat (Zhang et al., 2018) for multi-turn dialog generation. We follow the script of Luo et al. (2018) to pre-process the DailyDialog data, where the dialog is represented as request-response pairs. Then the first 80% pairs are used for training, the next 10% for validation, and the last 10% for test. We follow the script of Miller et al. (2017) to pre-process the Persona-Chat data, where the persona is inserted before the dialog.1 We only use the self original direction of Persona-Chat data in our experiments.
We use transformer small configuration for DailyDialog dataset and transformer base for Persona-Chat dataset, where both the encoder and the decoder consist of six layers. The (Embed Dim,FFN Embed Dim) of those configurations are (512, 1024) and (512, 2048), respectively. Our results are generated by beam search with beam size 5. We evaluate the BLEU (B1-B4), ROUGE-L (R-L), METEOR (Met.), and CIDEr metric (Papineni et al., 2002; Lin, 2004; Banerjee & Lavie, 2005; Vedantam et al., 2015). We use nlg-eval (Sharma et al., 2017) to compute the metrics except for BLEU scores, which are computed by the script from Luo et al. (2018). The experimental results of DailyDialog and Persona-Chat are shown in Table 1 and Table 2, respectively.
1The data process scripts and evaluation scripts are available in Appendix.
In Table 1 and Table 2, our method is first compared with conventional methods. In addition, we use recent data augmentation methods (e.g., SeqMix and SSMBA) as baselines. We also reproduce the results on Transformer and OR-NMT with sentence oracle. The subscript 1 of our method means we use single round augmentation.
From Table 1 we can see that our method is much better than existing works on all metrics, especially previous works of data augmentation. On all metrics mentioned above, we achieved propitious improvement over the Transformer baseline, especially for BLEU-4 (+3.57), R-L (+2.87), Met. (+3.28), and CIDEr (+34.99). The scores are also better than previous data augmentation methods. We can witness the similar improvement in Table 2, especially on the R-L (+0.93) and the CIDEr (+0.73) metric. Furthermore, following Liu et al. (2020b), we conducted the human evaluation on Persona-Chat data set, and the average score is improved from 1.18 to 1.49, which verified the quality improvement of our method. More details about human evaluation is in Appendix.

heading: 4.2 MACHINE TRANSLATION
text: On machine translation task, we conduct experiments on two widely used data sets: the IWSLT’14 English (EN) ↔ German (DE) data set (Cettolo et al., 2014), which is a small scale data set with 140k sentence pairs, and the WMT’14 EN→ DE dataset (Bojar et al., 2014), which is a larger one with 4.5M sentence pairs. We use the scripts from Ott et al. (2019) to process data. For IWSLT’14 EN↔DE, all words are lowercased and tokenized, and for WMT’14, all sentences are kept as original case then tokenized. Then the tokenized sentences are processed by BPE (Sennrich et al., 2016d) with 10k and 40k steps, respectively. We use transformer small for IWSLT’14 and transformer base for WMT’14, where the details are described in Section 4.1. The subscript 1, 2 of our method means the number of rounds used in augmentation. Our results are generated by beam search using beam size 5. For a fair comparison with previous works, we compute the BLEU score by the Moses script (Koehn et al., 2007) with the same tokenizer used by previous works.
The experimental results are summarized in Table 3 and Table 4, respectively. Our baselines include standard AR training without data augmentation, other Transformer variants, and previous works about data augmentation. Compared with Transformer without data augmentation, our method achieves 1.65-point BLEU gain on IWSLT’14 EN→DE direction and 2.36-point on DE→EN direction, and 1.8-point improvement on WMT’14 EN→DE translation (significance level 0.001). Besides, our results are better than other methods with complex architecture modification (the first group in tables) and source-side data augmentation (the second group in tables). These numbers clearly show the importance of target-side data augmentation and the potency of our algorithm.
Moreover, we added an extra round of data augmentation on the IWSLT’14 data set, and the results are in the last row of Table 3. We can see that the BLEU scores can be further boosted while the improvement is smaller than the first round’s. This phenomenon is consistent with other data augmentation methods, e.g., iterative back-translation (Hoang et al., 2018).
Finally, we combine our method with previous data augmentation method, i.e., back translation. More specifically, without using extra monolingual data, we translate the target sentences in the training set using a backward translation model. When jointly using BT and our method, the BLEU score can be boosted to 37.17, which is higher than using any of the methods solely. The result exemplifies that our method is complementary to other data augmentation methods.

heading: 4.3 ABSTRACTIVE SUMMARIZATION
text: We conduct experiments on the CNN/DM (Hermann et al., 2015) news summary dataset. To measure the effectiveness of our method with pre-trained models, We fine-tune the BART (Lewis et al., 2019) model on the CNN/DM dataset. Our experiments are based on BART.Large configuration. We use the scripts from Lewis et al. (2019) to process the CNN/DM data and the files2rouge tool to evaluate the ROUGE-1 (R-1), ROUGE-2 (R-2), and ROUGE-L (R-L) score. We report both scores and 95% confidence intervals. Our results are in shown Table 5.
In addition to BART (Chen et al., 2021), we also compare our method with previous works on this task and the scheduled sampling method OR-NMT with sentence-level Oracle. From the numbers in Table 5, our method significantly improves the ROUGE score of the BART model with 0.6- point R-1 and 0.7-point R-L and is better than other architecture changes or fine-tuning techniques. Moreover, our results are better than the scheduled sampling methods OR-NMT on sentence-oracle, demonstrating our augmentation method’s effectiveness.

heading: 4.4 ABLATION STUDY
text: Study of the parameter α: α is a key hyper-parameter of our system, which controls the augmented data weight and mix-up ratio. To study the effect of parameter α, we conduct experiments on the IWSLT’14 En → De data set. The Table 6 provides the results of this ablation study. In addition to a fixed value, we also explore two value schedule functions that change the α by epoch: (S1) α = min(1, 1.5/ √ log(EPOCH) + 1); (S2) α = (EPOCH − 55)2/3000 + 0.027(EPOCH < 55). We have more explanations about the design intuition of these two formulas in the Appendix.
First, we can observe that using a moderate value gives the best results. If the value is too high, e.g., 0.9, the results are negatively affected. Meanwhile, the best α for IWSLT’14 is around 0.3 and for Persona-Chat it is around 0.4. For the scheduled approach, different from previous works, we did not notice significant improvement. These results may suggest that a fixed value is already good enough for our algorithm.
Study of the consistency loss: We conduct experiments on the IWSLT’14 En → De data set to study the importance of consistency loss by changing the values of hyper-parameter β. As can be seen from Table 8, when the consistency loss is not used (β = 0), the BLEU score is improved by 0.9 point over the baseline without augmentation, which is lower than our best result. Meanwhile, the default value β = 1 gives the best result. When the value is too high or too low, the BLEU score will be negatively affected.
Study of the softmax temperature T : The softmax temperature T is an important hyperparameter of our algorithm. Empirically, we find a linear correlation between the best T value and vocabulary size and the detailed numbers are in Appendix. In Figure 3, for all data sets we use, we show the vocabulary size on the x-axis and the best T value on the y-axis. As the line chart shows, the best T value linearly decreases with the increase in vocabulary size. We suspect the reason is that the high temperature will smooth the probability distribution. Therefore, we suggest adjusting this parameter according to the vocabulary size to achieve the best results.
Study of the divergence D: Theoretically, we can use any function D in Equation 4 to measure the divergence between two distributions. We empirically choose Kullback–Leibler
divergence (DKL) or Jensen–Shannon divergence (DJS) and conduct experiments on the IWSLT’14 EN→ DE data set. We explore both single round augmentation and double rounds augmentation. It can be seen from the data in Table 9 that the DKL gives better results than JS divergence. Our intuition is that the synthetic data is not symmetric to real data therefore KL is more suitable here.
Iterative data augmentation: As described in Section 3, our method can be easily extended to iterative augmentation. We conduct experiments to evaluate the improvement on the IWSLT’14 DE→EN data set, and the results are in Table 10. The zero iteration means no aug-
mentation data is used. The numbers clearly show that the first iteration gives the most improvement, and other rounds have fewer yet positive effects on performance.

heading: 5 CONCLUSION AND FUTURE WORK
text: This paper studies a fundamental yet widely overlooked problem: augmenting the target input information for autoregressive sequence generation tasks. While data augmentation methods have been approved to be critical for machine learning, the study of enhancing the target-side conditional input is still limited. Meanwhile, we propose a sequence-level data augmentation method by using soft labels from the decoder output. Then the mixup method is used to generate synthetic data for model training. We conducted comprehensive experiments on three sequence generation tasks of five data sets. The experimental results suggest that augmenting target input benefits the sequence generation quality. Moreover, our method can vastly boost the model performance for these tasks. We hope this can bring more attention to this type of augmentation method.
There are many potential directions for future research. First, the investigation into other methods to augment this information will be fascinating. Second, applying such an idea to large-scale pretraining could also be substantial. Last, how to combine this method with unlabeled data is also a significant challenge. Therefore, we expect more research to be done in this area in the future.

heading: A DATA PROCESS AND EVALUATION SCRIPTS
text: The scripts for data preprocessing and evaluation can be found in the following URLs.
DailyDialog: https://github.com/lancopku/AMM
Persona-Chat: https://github.com/facebookresearch/ParlAI/tree/main/ projects/personachat
IWSLT’14: https://github.com/pytorch/fairseq/blob/main/examples/ translation/prepare-iwslt14.sh
WMT’14: https://github.com/pytorch/fairseq/blob/main/examples/ translation/prepare-wmt14en2de.sh
CNN/DM: https://github.com/pytorch/fairseq/blob/main/examples/ bart/README.summarization.md
Moses BLEU: https://github.com/moses-smt/mosesdecoder/blob/master/ scripts/generic/multi-bleu.perl
Statistical significance: https://github.com/moses-smt/mosesdecoder/blob/ master/scripts/analysis/bootstrap-hypothesis-difference-significance. pl
files2rouge: https://github.com/pltrdy/files2rouge
B DESIGN OF THE SCHEDULE FORMULA OF α
In Section 4.4, we mentioned two α schedule formula. Our intuition is that the value should decrease with the training progress, while the initial model quality may not be good, which will affect the synthetic data quality. Thus, we believed it could be better to have a warm-up stage when the model only learns from real data. Empirically, we find that the performance of the model with almost all hyper-parameters do not increase any more after the 50th epoch. That’s the reason that we choose 55 in S2. However, compared with a fixed value, we did not observe improvement. Therefore, we kept a fixed value in all other experiments.
Figure 4 contains the plots of these two functions.

heading: C TRAINING CURVE ABOUT THE CONSISTENCY LOSS
text: We show the training curve w/ and w/o consistency loss in Figure 5. As it shown in the figure, without consistency loss, the model will converge slower and the performance is worse.

heading: D DETAIL RESULTS OF THE EFFECT OF TEMPERATURE T
text: The results of different temperature on IWSLT and Persona-Chat are shown in Table 11 and Table 12.

heading: E STUDY OF ITERATIVE AUGMENTATION
text: We show the training curve of IWSLT’14 dataset for different iteration round in Figure 6.
The result shows than all augmented system is better than the baseline, while more iterations only give marginal improvement.

heading: F HUMAN EVALUATION DETAILS
text: We recruited 5 evaluators to judge on 100 samples in blind testing. For each response, each evaluator needs to give a score from 1 (low quality) to 4 (high qualiy). The score details are shown in Table 13.

Review summaries:
This paper experiments with a target-side sequence-level data-augmentation scheme for sequence-to-sequence generation tasks. The primary contribution of this work is an algorithm that leverages model-outputs to construct pseudo-target-side tokens (and consequently pseudo-sequences) for augmentation. It is done by incorporating soft-embeddings in the encoder-decoder model, a standard cross-entropy loss function supercharged with a consistency loss objective. One of the critical highlights is that this work doesn't require any external model for performing data augmentation. A comprehensive set of experiments on three sequence generation tasks are conducted to highlight the effectiveness of using such an approach for data augmentation.

Traditionally, the decoder of Seq2Seq model takes ground truth words of previous steps as input during training, while at inference, its input are those generated tokens starting from scratch. Thus there exists discrepancy between training and inference.  This paper presents an approach of data augmentation for the input of the decoder during training.  Specifically, the authors still feed the ground truth to the decoder, and then obtain its output vocabulary distributions that are multiplied with word embedding weights to get "pseudo tokens".  The contribution of this work comes from narrowing the gap of decoding procedure between training and testing.

This paper introduces a simple technique for target-side data augmentation. The high-level idea is to generate a "soft token" embedding by interpolating the target side word embeddings, with the output distribution generated by an initial model being the interpolation weight (after being adjusted by a temperature parameter $T$). The resulting training loss is in three terms: (1) original cross-entropy loss; (2) cross-entropy loss with the soft token embedding as the input; (3) a consistency loss controlling the distribution divergence of (1) and (2).  Results show that the proposed data augmentation method is very helpful for the dialog generation task, while also being somewhat helpful with neural machine translation for abstractive summarization. Ablation studies show that the weight of (1) cannot be too low, and that the optimal temperature parameter has some negative correlation with the vocabulary size.

This paper focuses on sequence generation tasks and proposes to perform data augmentation on the target side. During training, given an input-output pair ($x$, $y$), their model first uses teacher-forcing to get the output probability $p_y$ and then gets an augmented pair ($x$, $\tilde{y}$) by using "mixup" to mix the embeddings of $y$ and $p_y$. They also propose to encourage the consistency of predictions between ($x$, $y$) and ($x$, $\tilde{y}$) by minimizing the KL divergence of their output probabilities.  They experiment on three sequence generation tasks and demonstrate improvements over baselines.