Target-Side Input Augmentation for Sequence to Sequence Generation
Authors: Shufang Xie
Ang Lv
Yingce Xia
Lijun Wu
Tao Qin
Tie-Yan Liu
Rui Yan
Venue: ICLR 2022 Poster (2022)
Keywords: Sequence Gerneration
Data Augmentation

Abstract: Autoregressive sequence generation, a prevalent task in machine learning and natural language processing, generates every target token conditioned on both a source input and previously generated target tokens. Previous data augmentation methods, which have been shown to be effective for the task, mainly enhance source inputs (e.g., injecting noise into the source sequence by random swapping or masking, back translation, etc.) while overlooking the target-side augmentation. In this work, we propose a target-side augmentation method for sequence generation. In training, we use the decoder output probability distributions as soft indicators, which are multiplied with target token embeddings, to build pseudo tokens. These soft pseudo tokens are then used as target tokens to enhance the training. We conduct comprehensive experiments on various sequence generation tasks, including dialog generation, machine translation, and abstractive summarization. Without using any extra labeled data or introducing additional model parameters, our method significantly outperforms strong baselines. The code is available at https://github.com/TARGET-SIDE-DATA-AUG/TSDASG.

Published as a conference paper at ICLR 2022
TARGET-SIDE INPUT AUGMENTATION FOR SEQUENCE TO SEQUENCE GENERATION
Shufang ;ie Lv"; Yingee Xia", Lijun Wu², Tao Qin", Tie- Yan Liu", Rui Yan'l
1 Gaolling School of Artificial Intelligence, Remin University China
2 Microsoff Research Asia
'shufangxie@ruc. edu. cn, Lvangupup@email. com, zuiyanêruc, edu. on 2 [[yingee. xiia, lijuwu, taoqin, tyliujemicrosoft.
ABSTRACT
Autoregressive sequence generation, a prevalent task in machine learning and natural language processing. generates every target token conditioned on both a source input and previously generated target tokens. Previous data augmen- tation methods, which have been shown to be effective for the task, mainly en- hance source inputs (e.g., injecting noise into the source sequence by random swapping or masking, back translation, etc.) while overlooking the target-side augmentation. In this work, propose target-side augmentation method for sequence generation. In training, we use the decoder output probability distri- butions soft indicators, which are multiplied with target token embeddings, build pseudo tokens. These soft pscudo tokens are then used as target tokens to en- hance the training. We conduct comprehensive experiments generation tasks, including dialog generation, machine translation, and abstractive summarization. Without using any extra labeled data introducing additional model parameters, our method significantly outperforms strong baselines. The code available https:: https://github: com/ TARGET SIDE -ATTA TSDASc
various sequence
INTRODUCTION
Sequence generation is an important class of machine learning tasks and has made great progress
in recent years /Vaswani al. 2017: Hassan al. 2018 Zhou al. 2020 Stiennon et al.) 2020). In general, to generate sequences of interest based on a given source input. For example, in text summarization, the source input long document while the target sequence is a summary. In machine translation, the source input is a sentence in language while the target output is a sentence another language. Autoregressive (AR) methods, which generate tokens one by one au- toregressively to leverage previously generated tokens, are widely used in most sequence generation problems (Gat & Krahmer 2018 Brown et al. 2020).
Data augmentation is crucial technology for many machine learning tasks including sequence generation. The goal of data augmentation
increase the number of training examples without incurring additional efforts of manually labeling, which could be costly even impossible. For example, WWan et al. (2018) add random noise into input sequences, Gao et al. (2019) use pre- trained language model to synthesize pseudo input sequences, and Edunov et al. (2018) use back- translation to synthesize pseudo-pairs for machine translation. Furthermore, we list more works about the data augmentation method in Section
While these works have demonstrated promising improvements, an inherent aspect of AR sequence generation is generally overlooked. As illustrated in Figure in AR generation, each token is gen- erated based on two kinds of input information: the original source inputs and previously generated target-side tokens (aka target-side inputs). While most existing data augmentation works focus on enhancing source inputs, the augmentation of target-side inputs is under-explored. As far as we
"Equal contribution. "orresponning author: Rui Yan (ruiyan @ @du.ccd..c)..

---

Published as a conference paper at ICLR 2022
know, only there only few works (Mihaylova & Martins 2019 Zhang et al. 2019) weakly
related to this problem, which are to address the exposure bias issue Bengio et al. 2015 of the teacher forcing training. Those works adjust the sequence in token-level by randomly sampling target-side words from ground truth and model output. However, the token level adjustment may not fully leverage the potential of target input.
This work focuses on target-side data aug- mentation for sequence generation and pro- poses a sequence-level target-side augmenta- tion method. In training, we first feed the ground truth target tokens to the decoder to gen- erate the output probability distributions (over the target vocabulary) for each target position. Then these distributions are multiplied with the target-side token embeddings to build soft pseudo tokens as enhanced target-side inputs. Inspired by the success of mixup (Hongyi al. 2018), the pseudo tokens are further mixed up with original ground truth target tokens to build synthetic target-side sequences. Next, our model is trained with both ground truth data and synthetic data. Note that we only change the target tokens for decoder input rather than target outputs labels. Our model can benefit from the stochastic representation of target input tokens achieve better performance. The inference process is kept the same as the conventional method. Moreover, no extra data model parameters are required during inference. That is, we do not introduce any additional burden model inference.
Target
sunny
ENCODER
DECODER
What's the weather today?
<BOS>
Source
Target Input
Figure 1: Illustration of autoregressive generation.
We conduct experiments on various sequence generation tasks, including dialog generation, machine translation, and abstractive summarization. On five datasets, our method outperforms standard AR training without data augmentation, as well as existing data augmentation methods.
Our main contributions are follows:
Problem: We shine light on previously neglected problem, namely, the target-side inputs ar- gumentation (TIA) for autoregressive sequence generation.
We propose a new algorithm, the Soft Sequence-level Target-side Inputs Argumenta- tion (SSTIA), which constructs synthetic target-side input tokens by generating soft pscudo-token sequences.
Method:
Experiment: We test our method on three sequence generation tasks with five data sets. Our method achieves good improvements side data augmentation and the effectiveness of our algorithm. We want to call more attention from the community to design better methods for target-side data augmentation.
strong baselines, which reveals the importance of target-
RELATED WORK
Data augmentation has been widely used in almost tasks of deep learning and has been widely surveyed before (Shorten & Khoshgoftaar [019 Wen et al. 2021 Feng et al. (202)) }. In this section, we only briefly discuss works related our method.
Rule-based augmentation: Some works propose to use heuristic rules to augment the source data. For example, WWan et al. (2018) randomly replace words in sentences with another word in the vocabulary, Guo et al. (2020) softly combine sequences in the training set, Shen et al. (2020) randomly erase part of sequence build restricted views, and Sennrich et al. (016b) pervasive dropout to enrich data. However, those methods target to enhance source input and do not pay attention to the target input.
2

---

Published as a conference paper at ICLR 2022
Model-based augmentation: In this category of methods, deep learning models used gen- erate the augmented data. For example, Gao et al. (2019) leverage a language model to enhance the source tokens, and Kumar et al. (2020) use BERT (Devlin et al.] 2019). GPT-2 (Radford et al. 2019), and BART (Lewis et al. 2020) to generate pseudo data. For tasks with high duality, e.g.. machine translation, back translation is widely used (Sennrich al., 2016a] Edunov et al. 2018 Prabhumoye et al. 2018 Hoang et al. 2018). Usually, large-scale monolingual data is used to build pseudo training data. Similar rule-based methods, the model-based methods do not pay much effort to the target input either.
Knowledge distillation: Knowledge distillation is also a widely used method for sequence gen- eration (Hinton et al.) [2015] Kim & Rush, 2016 He et al., 2019 Gou et al., 202T). In this method, teacher models are trained to generate synthetic labels for student models. However, for student models, the target input and output of the decoder are still same. On the contrary, our methods focus on changing the target input, therefore the decoder input and output are different. And we do not have any intention of model compression.
Scheduled sampling: Scheduled sampling methods are invented to solve the exposure-bie prob- lem (Bengio al. 2015 Mihaylova & Martins 2019 Zhang et al. 2019). Although these methods also change the target input, they can only work in the token-level adaptation. In contrast, our method builds sequence-level synthetic data improve training performance. This is an important difference between our method and scheduled sampling methods. Besides, when using sentence- level oracle, OR-NMT (Zhang al. 2019) need full decoding and training time could be 5æ slower.
Reinforcement learning: The refinement learning (RL) based sequence generation meth- ods (Ranzato al. 2015, Shen et al. 2016, Norouzi et al., 2016) can directly optimize the metrics that are used in test time and the target input of decoder is not always necessary. However, success- fully training a RL-based sequence generation is still challenging due to its instability (Wu et al., 2018). Therefore, we focus on AR generation in this paper.
Non-autoregrresive generation: Non-autoregressive (NAR) generation has been proposed as an alternative sequence generation paradigm, where all target tokens are generated in parallel. Although model performance has been greatly improved by many recent works (Gu et al. 2018 Qian et al.) 202 Ren al.) (020), the AR generation is still the most common approach. Therefore, we choose to study the AR generation in this work and leave the application on NAR for future work.
Iterative refinement: Iterative refinement is common way improve performance. For exam- plc,[iaa et al. (2017) proposed deliberation networks, and Lee et a (2018) proposed to use iterative refinement in the NAR generation (Gu al. 2018). However, these methods need multiple passes during inference time. Therefore, the iterative NAR is slower than fully NAR method (Gu & Kong 202TI. By contrast, our method only augments the data during training and has no extra inference cost. This makes our methods applicable to more scenarios where latency is critical.
3
OUR METHOD
In this section, we first introduce the background and notations. Next, we present the target-side data augmentation framework and our proposed method. Finally, we will discuss the relation with iterative data augmentation methods.
Background: Let æ denote the source-side input, and let y (v1 32.
V/o denote target
sequence to generate, where n is the sequence length, and each yı belongs to predefined vocabulary made up of all words or sub-words. Let D {(x', y ) denote the training set with size M. In the AR generation, our task is model the distribution P(y]z). Specifically,
1 II P(y lr, Ves)
P(y|x)
where yer denotes the uub-sequence before y: and yo is represented by a special begin-of oe-sentence (BOS) token. We can that the probability of generating yi conditions both and yei.
3

---

Published as a conference paper at ICLR 2022
Conventional source-side data augmentation methods can be summarized using function faug to generate enhanced data set. Specifically,, for any (æ), 3²) D, generate the source-side sequence by 3 =Jaug (x' yy) and append (ø), v²) ) to the enhanced dataset D. The faug can also solely base on z or y. This augmentation method can be rule based (e.g.. a noise function) or model based (e.g.. pre-trained language model). It may also contain some external knowledge heuristic (Wang
et al. 2018 Kobayashi 2018 Kumar et al. 2020).
Based on the original dataset and the augmented dataset D, the training objective function is defined as follows:
== aLn + - a
M
M
Σ
log Ply ((y'(z²)~ (1 a)
Σ
log P(v"
= - M
M
M
n4 log Plusl=', v'zz) a M
n
= - a)
ΣΣ
log Plyl v22)),
M
1=
The parameter a in Equation n2 denotes the length of yı-
2 balances the loss weight on real data D and synthetic data D, and the
Enhancing the target-side conditional input: the source-side input, we propose a new method that augments the previously generated target to-
Different from Smg which focuses enhancing
kens, i.e., the yei in equation 1 We create an augmented training data set = {(x*, y } km
where the ÿ is the augmented sequence serving as the conditional input of the target sequence. The workflow shown in Figure a).
The loss function on D is defined follows:
Lp = M
Σ
log (wlfko;$22))
j=1
2020b Shen al. 2020) have found that maximizing agreement between multiple augmented data that are generated from single piece of data can improve model performance. Inspired by this, we design consistency loss L. between synthetic data and real data to stabilize the training. It is defined as
Some recent study (Chen et al.,
M D(P(-E'' P(Ya', (22)).
Le - M
(4)
i=1==
The D in Equation can be any function to measure the divergence of two distributions, e.g., Kull- back-Leibler divergence Jensen-Shannon divergence. We found that using L. stabilize the training.
Our final loss function is defined as:
C = - (1-a)Lp + aL + 3Lc
(5)
Empirically, we found 8 = 1 is good default value and used in all experiments.
Augmented data generation: We use model-based method to generate the augmented sequence y. However, do not need external pre- pre-trained models like (Gao et al. 2019). Specifically,
(1) We obtain the logits iș for any f1,2, mi), which are the values before softmax, of all target tokens. Intuitively, decoder of the sequence generation model and i3 is a [Vi-dimensoonal vector. Note that we feed the groundtruth yej into the decoder to obtain it.
dec(enc(x)) v2y)), where enc and dec denote the encoder and
(2) We obtain the Vj as follows:
exp(is/TT
+ (1 - >)u? W Wemb:
(6)
M vil exp((/(]/TT)
4

---

Published as a conference paper at ICLR 2022
(a)
(b)
0.6
Lusea
0.1
0.7
Ground
Soft Labels
0.1
0.0
0.1
Byrthitc Prudiction
|
Consistency Loss
0.0
0.1
0.2
Encoder
Decoder
Synthetic
Source Input
Ground Truth Target
Augmanted Data
Synthotic Data
Ground Truth Target
Figure 2: Illustration of algorithm. (a) Target side data augmentation. The decoder consumes both ground truth and synthetic data as target input. (b) Augmented target data generation with soft labels. The @ denotes mixup operation.
where (i) W Emb the target word embedding with size [ x d; (ii) is the softmax temperature; (iii) the square brackets denotes the subscription of vector; (iv) y is the mixup ratio and we
the same value as a Equation for all experiments. The y3 is a mixture of two parts: (i) the embedding of the groundtruth yÿ: (ii) soft representation, which is the product between the target embedding and distribution output by a sequence generation model. Empirically, we found T is an important modern sequence generation follows the encoder-decoder architecture we can further reduce the
and more details can be found in Section 4.4 In addition, as the
augmentation cost by reusing the encoder output because it is not changed during decoding.
Multi-round enhancement: Similar to iterative back-translation (Hoang et al. 2018 our model be easily extended multiple rounds. That is, we the Equation 6 multiple times, where the logits need to be replaced by the previous output. The experimental results suggest that many iterations only give marginal extra gain. Therefore, our experiments mainly focus on single round augmentation.
EXPERIMENT
We conduct experiments on various sequence generation tasks. The experiment details and results are in the following sections. Our implementation is based on the FairSeq framework (Ott et al., 2019). We use Transformer (Vaswani al., 2017) network architecture in all experiments with different model sizes, which are adjusted according to the data size. During training, we use Adam optimizer (Kingma & Ba 2015) with Adam / (0.9, 0.98) and invert_sqrt. learning rate scheduler. Meanwhile, we used label smoothing of value 0.1, which is the same value as Vaswani ctt al. 2017).
4.1 DIALOG GENERATION
commonly used dialog generation data sets: the DailyDialog Lii et al. 2017) for single-turn dialog generation and Persona-Chal (Zhang et al.) 2018) for multi-turn dialog generation. We follow the script of Luo al. (2018) to pre-process the DailyDialog data, where the dialog is represented as request-respons, pairs. Then the first 80% pairs are used for training. the next 10% for validation, and the last 10% for test. We follow the script of Miller al. (017)) to pre-process the Persona-Chal data, where the persona is inserted before the dialog] We only use the self original direction of Persona-Chat data in our experiments.
We conduct experiments
We use trans forfooreeeee for Persona-Chal dataset, where both the encoder and the decoder consist of six layers. The (Embed_Dim, PFN_Embed..m
11 configuration for DailyDialog dataset and trans former base
of those configurations are (512, 1024) and (512, 2048), respec- tively. Our results are generated by beam search with beam size We evaluate the BLEU (B1-B4),
"The data process scripts and evaluation scripts available Appendix.

---

Published as a conference paper at ICLR 2022
ROUGE-L (R-L), METEOR (Met), and CIDEr metric Papineni et al. 2002: Lin 2004 Banerjee & Lavie 2005 Vedantam et al. 2013). We use nlg-eval (Sharma et al. 2017) to compute the
metrics except for BLEU scores, which computed by the script from Luo et 20188 The
perimental results of DailyDialog and Persona-Chal are shown in Table and Table respectively.
Table 1: Experimental results on DailyDialog data set.
Method
B1
B2
B3
B4
R-L
Met.
CIDEr
Seq2Seg (Luo et al. 2018)
12.43
4.57
2.69
1.84
AEM (Luo al. 2018)
13.55
4.89
3.04
2.16
Transformer (Our reproduce)
14.70
6.13
4.15
3.13
17.45
5.36
18.58
Seq Guo et al. 2020)
12.09
4.16
2.50
1.81
14.96
5,46
6.68
SSMBA (Ng et 2020)
12.96
4.68
3.03
2.34
15.69
4.43
10.23
OR-NMT Zhang et al. 2019)
16.18
7.94
6.13
5.21
18.86
6.78
38.94
SSTIA (Ours)
17.53
9.32
7.54
6.70
20.32
8.64
53.57
In Table and Table our method is first compared with conventional methods. In addition, use recent data augmentation methods (e.g., SeqMix and SSMBA) as baselines. We also reproduce the results on Transformer and OR-NMT with sentence oracle. The subscript of our method means we use single round augmentation.
From Table I we can see that our method is much better than existing works on all metrics, espe- cially previous works of data augmentation. On all metrics mentioned above, we achieved propi- tious improvement over the Transformer baseline, especially for BLEU-4 (+3.57), R-I. (+2.87), Met. (+3.28), and CIDEr (+34.999). The scores are also better than previous data augmentation methods'. We can witness the similar improvement in Table 2 especially on the R-L (+0.93) and the CIDEr (+0.73) metric. Furthermore, following Liu et al. (2020b), we conducted the human eval- uation on Persona-Chat data set, and the average score is improved from 1.18 to 1.49, which verified the quality improvement of our method. More details about human evaluation is in Appendix.
Table 2: Experimental results Persona Massona-Chatto dataset.
Method
B1
B2
B3
B4
R-L
Met.
CIDEr
Transformer TTy et al. 2021)
3.20
13.38
5.89
18.94
Transformer (Our reproduce)
18.33
8.28
4.99
3.44
19.27
8.13
20.42
SeqMix Guo et al. 2020)
18.11
8.30
3.43
2.83
18.98
8.32
20.26
SSMBA gg et 2020)
18.69
7.07
3.90
2.72
19.28
8.30
11.66
OR-NMT Zhang et al.
2019)
18.69
8.34
4.97
3.42
19.80
8.43
20.14
SSTIA (Ours)
19.03
8.68
5.21
3.55
20.20
8.43
21.15
4.2
MACHINE TRANSLATION
On machine translation task, we conduct experiments on two widely used data sets: the IWSLT' 14 English (EN) ++ German (DE) data set (Cettolo et al. 2014), which is a small scale data set with 140k sentence pairs, and the WMT 14 EN + DE dataset (Bojar et al., 2014), which is a larger one with 4.5M sentence pairs. We use the scripts from Ou et (201) process data. For IWSLT' 14 EN+DE all words are lowercased and tokenized, and for WMT 14, all sentences are kept as orig- inal case then tokenized. Then the tokenized sentences are processed by BPE (Sennrich et al.) 2016d) with 10k and 40k: steps, respectively. We use tran forsformeee rmer lammerrsmaallll for IWSLT"14 and trans cotommerbaass for WMT' where the details are described in Section[.]] The subscript 1 or 2 of our method means the number of rounds used in augmentation. Our results are generated by beam search using beam size 5. We compute the BLEU score by the Moses script (Kochn et al.) 2007) with the same tokenizer used by previous works.
The experimental results are summarized in Table Table respectively. Our baselines include standard AR training without data augmentation. other Transformer variants, and previous works about data augmentation. Compared with Transformer without data augmentation, our method achieves I. 1.65-t BLEU gain on IWSLT" 14 DE direction, 2.36-point on DE- direction,

---

Published as a conference paper at ICLR 2022
and 1.8- 1.8-point improvement WMT' EN-DE translation (significance level 0.001). Besides, our results are better than other methods with complex architecture modification (the first group in tables) and source-side data augmentation (the second group in tables). These numbers clearly show the importance of target-side data augmentation and the potency of our algorithm.
Moreover, added extra round of data augmentation the IWSLT" 14 data set, and the results are in the last row of Table We can see that the BLEU scores can be further boosted while the improvement smaller than the first round's. This phenomenon consistent with other data augmentation methods, e.g., iterative back-translation (Hoang et al. 2018).
Finally, combine our method with previous data augmentation method, i.e., back translation. More specifically, without using extra monolingual data, we translate the target sentences in the training set using a backward translation model. When jointly using BT and our method, the BLEU score can be boosted to 37.17, which is higher than using any of the methods solely. The result exemplifies that our method is complementary to other data augmentation methods.
Table 3: Experimental results on IWSLT English ++ German translation task.
Method
DE +NN
Transformer Vaswani et al.
2017
28.57
34.40
DynamicConv (Wu et al. 2019
28.70
35.00
MAT (Fan et al. 2020
29.90
35.59
Adversarial training Wang et al.
2019)
28.43
35,18
Wood (Sennrich et al. 2016c
29.20
35.60
SCA (Gao et al. 2019
35.78
SwitchOut Wang et a..
2018)
29.00
35.90
SSMBA (gg al. 2020)
36.10
SeqMix (Guo et al. 20200)
29.50
36,20
Back Translation (Our reproduce)
36.38
Mixed Representations (Wu et al. 2020)
29.93
36.41
SSTIA (Ours)
30.14
36.47
SSTIA (Ours)
30.22
36.76
SSTIA + Back Translation
37.17
Table 4: Experimental results on WMT 14 English >German translation task.
Method
BLEU
Transformer (Vawwani et
2017
27.3
Admin (Liu et al. 2020a
27.9
Evolved Transtormer (So et al.
2019
28.4
Weighted Transformer (Ahmed et al.
2017)
28.4
Adversarial Training (Wang al.
2019)
28.4
OR-NMT (Zhang et al. 2019)
28.5
Cransformer+Cutott (Shen et al.
2020
29.1
SSTTA, (Ours)
29.1
4.3 ABSTRACTIVE SUMMARIZATION
We conduct experiments on the CNN/DM (Hermann et al. 2013) news summary dataset. To mea- sure the effectiveness of our method with pre-trained models, We fine-tune the BART (Lewis et al.) 2020) model on the CNN/DM dataset. Our experiments are based on BART La rge configuration. We use the scripts from Lewis et al. (2020) to process the CNN/DM data and the files2rouge tool to evaluate the ROUGE-I (R-1), ROUGE-2 (R-2), and ROUGE-L (R-L) score. We report both scores and 95% confidence intervals. Our results are shown in Table 5
In addition to BART (Chnn et al. 2020a), we also compare our method with previous works on this task and the scheduled sampling method OR-NMT with sentence-level Oracle. From the numbers in Table 5 our method significantly improves the ROUGE score of the BART model with 0.6- point R-1 and 0.7-point R-L. and is better than other architecture changes fine-tuning techniques.

---

Published as a conference paper at ICLR 2022
Table 5: Experimental results CNN/DM dataset.
Method
R-1
R-2
R-L
BART Lewis et al.
2020
44.16
21.28
40.90
BART + OR- Zhang ct al.
2019
44.01
21.26
40.81
PEGASUS (Zhang al. 2020
44.17
21.47
41.11
ERNIE-GEN (Xiao et al. 2020)
44.02
21.17
41.26
ProphetNet (Qi et al. 2020)
44.20
21.17
41.30
BART + R3F (FFFFAAghaaaaa et al. 2021)
44.38
21.53
41.17
BART + , (Ours)
44.76
21.46
41.57
95% confidence interval
(44.54, 44.95)
(21.23, 21.68)
(41.36,
Moreover, results better than the scheduled sampling methods OR-NMT sentence-oracle, demonstrating our augmentation method's effectiveness.
4.4 ABLATION STUDY
of our system, which controls the aug- mented data weight and up ratio. To study the effect of parameter we conduct experiments on the IWSLT En De data set. The Table Table T0 In addition to fixed value, we also explore two value schedule functions that change the by epoch: (S1) a-) min 1.5/ We have more results on other datasets and explanations about the design intuition of these two formulas in the Appendix.
Study of the parameter a: is key
the results of this ablation study.
1); (S2) (EPOCH /3000 + 0,027(BPOCII < 55).
First, we can observe that using moderate value gives the best results. If the value is too high, c.g., 0.9, the results are negatively affected. For the scheduled approach, different from previous works, we did not notice significant improvement. These results may suggest that a fixed value is already good enough for our algorithm.
Table 6: Ablation study of a on IWSLT" 14 dataset.
0.1
0.3
0.4
0.5
0.7
0.9
S1
$2
(ttt(test)
36.43
36.47
36.43
36.40
36.22
36.19
36.00
36.38
BLEU(dev)
36.75
36.78
36.74
36.71
36.68
36.53
36.49
36.57
Study of the consistency loss: We conduct experiments on the IWSLT' En De data set to study the importance of consistency loss by changing the values of hyper-parameter B. As can be seen from Table 7 when the consistency loss is not used (8 0), the BLEU score is improved by 0.9 point over the baseline without augmentation, which is lower than our best result. Mcanwhile, the default value gives the best result. When the value is too high or too low, the BLEU score will be negatively affected.
Table 7: Ablation study of / in loss function.
B
0
0.6
0.8
(tes(
35.28
36.03
36.29
36.47
36.32
36.30
(lEU(dev))
35.96
36.39
36.57
36.78
36.53
36.46
Study of the softmax temperature T: The softmax temperature T is an important hyper- parameter of our algorithm. Empirically, we find a linear correlation between the best T value and vocabulary size. In Figure for all data sets use, we show the vocabulary size the x-axis and the best T value on the y-axis (Detailed numbers are in Appendix). As the line chart shows, the best T value linearly decreases with the increase in vocabulary size. We suspect the reason is that the high temperature will smooth the probability distribution. Therefore, we suggest adjusting this parameter according to the vocabulary size achieve the best results.
8

---

Published as a conference paper at ICLR 2022
PersonaChat
IWSLT
DailyBialog
value
3
WMT
CNN/DM
10000
20000
30000
40000
50000
Vocabulary size
Figure 3: The best temperature on different vocabulary size.
Study of the divergence D: Theoretically,
Table Ablation on the choice of D.
can use any function D in Equation to measure the divergence between two distribu- tions. We empirically choose Lulbback-Leibler divergence (DKL.) Jensen-Shannon diver- gence (Dis) and conduct experiments on the IWSLT' 14 EN- DE data set. We explore both single round augmentation and double rounds augmentation. It can be seen from the data in Table 8 that the DKL. gives better results than JS di-
Double Rounds Single Round
(test)
36.47
36.76
DRI (dev)
36.78
37.19
(ts(test)
35.95
36.50
Dis(dev)
36.34
36.91
vergence. Our intuition that the synthetic data is not symmetric to real data therefore KI. is more suitable here.
Iterative data augmentation: As described in Section 3 our method can be easily ex- tended to iterative augmentation. We conduct experiments to evaluate the improvement on the IWSLT' DE ENN data set, and the results are iteration means aug-
Table 9: Experimental results on iterative data augmentation.
# Iterations
(tet( BLEU(dev) 35.49
34,40
36.47
36.76
36.77
in Table 9 The mentation data is used. The numbers clearly show that the first iteration gives the most improvement, and other rounds have fewer yet positive effects on performance.
36.78
37.19
37.18
CONCLUSION AND FUTURE WORK
This paper studies a fundamental yet widely overlooked problem: augmenting the target input infor- mation for autoregressive sequence generation tasks. While data augmentation methods have been approved be critical for machine learning, the study of enhancing the target-side conditional input is still limited. Meanwhile, we propose a sequence-level data augmentation method by using soft labels from the decoder output. Then the mixup method is used generate synthetic data for model training. We conducted comprehensive experiments on three sequence generation tasks of five data sets. The experimental results suggest that augmenting target input benefits the sequence generation quality. Morcover, our method can vastly boost the model performance for these tasks. We hope this can bring more attention to this type of augmentation method.
There are many potential directions for future research. First, the investigation into other meth- ods to augment this information will be fascinating. Second, applying such an idea to large-scale pre-training could also be substantial. Last, how combine this method with other mono data aug- mentation methods, e.g., a VAE model, is also a significant challenge. Therefore, we expect more research to be done in this in the future.

---

Published as a conference paper at ICLR 2022
ACKNOWLEDGEMENT
We would like to thank the anonymous reviewers for their insightful comments. This work was supported by National Natural Science Foundation of China (NSFC Grant No. 62122089 and No. 61876196), Beijing Outstanding Young Scientist Program NO. JUWZYJH01201110002222 and Intelligent Social Governance Platform, Major Innovation & Planning Interdiseiplinary Platform for the "Double-Firs. Class" Initiative, Renmin University of China. We also wish to acknowledge the support provided and contribution made by Public Policy and Decision-making Research Lab of RUC. Rui Yan is supported by Beijing Academy of Artificial Intelligence (BAAI).
REFERENCES
Armen Aghajanyan, Akshat Shrivastava, Anchit Gupta, Naman Goyal, Luke Zettlemoyer, and Sonal Gupta. Better fine-tuning by reducing representational collapse. In International Confer- ence on Learning Representations, 2021. (tlL:ttps:/// pectps::/oppereviiie net. ftt./forrmddd
OQC8SN70MIV.
Karim Ahmed, Nitish Shirish Keskar, and Richard Socher. Weighted transformer network for ma- chine translation, 2017.
Satanjeev Banerjee and Alon Lavie. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, pp. 65-72, 2005.
Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer. Scheduled sampling for sequence prediction with recurrent neural networks. Advances in neural information processing systems, 28, 2015.
Ondłej Bojar, Christian Buck, Christian Federmann, Barry Haddow, Philipp Koehn, Johannes Lev- cling, Christof Monz, Pavel Pecina, Matt Post, Herve Saint- Amand, et al. Findings of the 2014 workshop on statistical machine translation. In Proceedings of the ninth workshop on statistical machine translation, pp. 12-58, 2014.
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners, 2020.
Mauro Cettolo, Jan Niehues, Sebastian Stüker, Luisa Bentivogli, and Marcello Federico. Report on the 11th iwslt evaluation campaign, iwslt 2014. In Proceedings of the International Workshop on Spoken Language Translation, Hanoi, Vietnam, volume 57, 2014.
Daoyuan Chen, Yaliang Li, Minghui Qiu, Zhen Wang, Bofang Li, Bolin Ding, Hongbo Deng, Jun Huang, Wei Lin, and Jingren Zhou. Adabert: Task-adaptive bert compression with differentiable neural architecture search. Christian Bessiere (ed.), Proceedings of the Twenty-Ninth Inter- national Joint Conference on Artificial Intelligence, IJCAI-20, pp.
International Joint Conferences on Artificial Intelligence Organization, 2020a. doi: 00.224933jcai.2022000 URL
https://doo .rgg 10. 2496349933555 ijaii 2020/341 Main track.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pp. 1697-1607. PMLR, 2020b.
Jacob Devlin, Ming-Wei Wei Chang, Kenton Lee, and Kristina Toutanova. BERT; Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Volume (Long and Short Papers), pp. 4171-4186, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. doi: 10. 10.18653/v//N19-1 1423. URL https:
7/acl anchhnntholoooooooooo ogy. hww.hhhloooooor.g//11l///// N-
10

---

Published as a conference paper at ICLR 2022
Sergey Edunov, Myle Ott, Michael Auli, and David Grangier. Understanding back-translation at scale. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Pro- cessing, pp. 489-500, 2018.
Yang Fan, Shufang Xie, Yingee Xia, Lijun Wu, Tao Qin, Xiang-Yang Li, and Tie- Liu. Multi- branch attentive transformer, 2020.
Steven Y Feng, Varun Gangal, Jason Wei, Sarath Chandar, Soroush Vosoughi, Teruko Mitamura, and Eduard Hovy. A survey of data augmentation approaches for nlp. Findings ACI., 2021.
Fei Gao, Jinhua Zhu, Lijun Wu, Yingce Xia, Tao Qin, Xueqi Cheng, Wengang Zhou, and Tie- Yan Liu. Soft contextual data augmentation for neural machine translation. In Proceedings of the 57th Annual Meeting the Association for Computational Linguistics, pp. 5539-5544, Florence, Italy, July 2019. Association for Computational Linguistics. doi: 10. 10.18653/11/P19-1555 URL hEtps://aclantthoo.o
1555
Albert Gatt and Emiel Krahmer. Survey of the state of the art in natural language generation: Core tasks, applications and evaluation. Journal of Artificial Intelligence Reseurch, 61:65-170, 2018.
Jianping Gou, Baosheng Yu, Stephen J Maybank, and Dacheng Tao. Knowledge distillation: A
survey. International Journal of Computer Vision, 129(6):1789 1819, 2021.
Jiatao Gu and Xiang Kong. Fully non-autoregressive neural machine translation: Tricks of the trade. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp. 120-133, Online, August 2021. Association for Computational Linguistics. doi: 10.18653/v1/ 2021 wiw..iinng--al..111
/ttt:p:://claathhoooy..oooo
Jiatao Gu, James Bradbury, Caiming Xiong, Victor OK Li, and Richard Socher. Non-autoregressive neural machine translation. In International Conference on Learning Representattinss 2018.
Demi Guo, Yoon Kim, and Alexander Rush. Sequence-level mixed sample data augmentation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Process- ing (EMNLP), pp. 5547-5552, Online, November 2020. Association for Computational Linguis- tics. doi: 112.1666/////022.mmmllpp 447. URL htt.llttss//aaaaattllloooo org/2020. amnlp-mas in. 447
Hany Hassan, Anthony Aue, Chang Chen, Vishal Chowdhary, Jonathan Clark, Christian Federmann, William Lewis, Mu Li, Shujie Liu, Tie-Yan Liu, Renqian Luo, Arul Menezes, Tao Qin, Frank Seide, Xu Tan, Fei Tian, Lijun Wu, Shuangzhi Wu, Yingee Xia, Dongdong Zhang. Zhirui Zhang, and Ming Zhou. Achieving human parity on automatic chinese to english news translation. CoRR, 0ts//803300567, 2018. URL http:
Xuedong Huang, Marcin
//arxiv.orrg///o abs/ 803. 05567
Junxian He, Jiatao Gu, Jiajun Shen, and Marc' Aurelio Ranzato. Revisiting self-training for neural sequence generation. In International Conference on Learning Representations, 2019.
Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. Teaching machines to read and comprehend. Advances in neural information processing systems, 28: 18:1693-1701, 2015.
Geoffrey Hinton, Oriol Vinyals, and Jeffrey Dean. Distilling the knowledge in a neural network. In NIPS Deep Learning and Representation Learning Workshop, 2015. URL Uttt:p////aaxii
org/ abs/ 1 1503. 02531
Vu Cong Duy Hoang, Philipp Koehn, Gholamreza Haffari, and Trevor Cohn. Iterative back- translation for neural machine translation. In Proceedings of the 2nd Workshop Neural Ma- chine Translation and Generation, pp. 18-24, 2018
Zhang Hongyi, Cisse Moustapha, Dauphin Yann, N., and Paz David. mixup: Beyond em- pirical risk minimization. International Conference on Learning Representations, 2018. URL
https: ptttps://openrev// net www.:nee//oommmmldr.lgg...

---

Published as a conference paper at ICLR 2022
Yoon Kim and Alexander M. Rush. Sequence- knowledge distillation. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, 2016. doi: 10.18653/v1/
d16-1133 1139. URL/http: dx. doi. 10 http://....1.5///// 11t.1111/1//11111111111
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR), 2015.
Sosuke Kobayashi. Contextual augmentation: Data augmentation by words with paradigmatic rela- tions. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers). 2018. doi: 10.18653/v1/n188 2072.
URL / cx www. 1000.18653/000 2072
Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison- Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondřej Bo- jar, Alexandra Constantin, and Evan Herbst. Moses: Open source toolkit for statistical ma- chine translation. In Proceedings of the 45th Annual Meeting of the Association for Com- putational Linguistics Companion Volume Proceedings of the Demo and Poster Sessions, pp- 177-180, Prague, Czech Republic, June 2007. Association for Computational Linguisties. URL https: antho ogy. wwtppooggp00000000
Varun Kumar, Ashutosh Choudhary, and Eunah Cho. Data augmentation using pre-trained trans- former models, 2020.
Jason Lee, Elman Mansimov, and Kyunghyun Cho. Deterministic non-autoregressive neural quence modeling by iterative refinement. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 2018. doi: 10.18653/v1/d18- -1499
URL http:
fidx.doi.. org 0tte//10.1865///// 11111116///11--1111
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. BART: Denoising sequence-t0-scqueene pre- training for natural language generation, translation, and comprehension. In Proceedings of the 58th Anmual Meeting of the Association for Computational Linguistics, pp. 7871-7880, On- line, July 2020. Association for Computational Linguistics. doi: 10. URL https:://// //ac aantho org.oogg./oooooooooo...oo.oo.... 2020 acl imain. 703
Yanran Li, Hui Su, Xiaoyu Shen, Wenjie Li, Ziqiang Cao, and Shuzi Niu. DailyDialog: A man- ually labelled multi-turn dialogue dataset. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 986-995. Taipei, Taiwan, November 2017. Asian Federation of Natural Language Processing. URL hctps:
//aclanthol ogy. wtw.o//..///1100//999911
Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pp. 74-81, 2004.
Liyuan Liu, Xiaodong Liu, Jianfeng Gao, Weizhu Chen, and Jiawei Han. Understanding the diffi- culty of training transformers, 2020a.
Qian Liu, Yihong Chen, Bei Chen, Jian-Guang LOU, Zixuan Chen, Bin Zhou, and Dongmei Zhang- You impress me: Dialogue generation via mutual persona perception. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020b. doi: 1/.100 doi. wttp./101111/////2222220002 acl ma mai..ainn................m...... 131
acl-main. 131. htttt:
Liangehen Luo. Jingjing Xu, Junyang Lin, Qi Zeng, and Xu Sun. An auto-encoder matching model for learning utterance-level semantic dependency in dialogue generation, 2018
Tsvetomila Mihaylova and André T. Martins. Scheduled sampling for transformers. In Pro- ceedings of the 57th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop. pp. 351-356, Florence, Italy, July 2019. Association for Computational Lin- guistics. doi: 10.18653/v 11t33//1/1192200 1/P19-2049. URL http///clanntthht
wrt/olggg110 2049
Alexander Miller, Will Feng, Dhruv Batra, Antoine Bordes, Adam Fisch, Jiasen Lu, Devi Parikh, and Jason Weston. ParlAl: A dialog research software platform. In Proceedings of the 2017 Conference Empirical Methods in Natural Language Processing: System Demonstrations, pp-
12

---

Published as a conference paper at ICLR 2022
79 84, Copenhagen, Denmark, September 2017. Association for Computational Linguisties. doi: 10.18663/v1/D17--11 ottp://chee///aaaaattooooooooro.
Nathan Ng. Kyunghyun Cho, and Marzyeh Ghassemi. SSMBA: Self-supervised manifold based data augmentation for improving out-of-domain robustness. In Proceedings of the 2020 Conference on
Empirical Methods Natural Language Processing (EMNLP), pp. 1168-1283, Online, Novem- ber 2020. Association for Computational Linguistics. doi: 10.18653/v 00138653311222000 htt.:/..20mn/////. ea1.000.mmlpppma emnlp-main..9 main.97. URL https:://aaaanthllggggoooo/ hrtg emnip- main. 97
Mohammad Norouzi, Samy Bengio, Zhifeng Chen, Navdeep Jaitly, Mike Schuster, Yonghui Wu, and Dale Schuurmans. Reward augmented maximum likelihood for neural structured prediction, 2016.
Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. fairseq: A fast, extensible toolkit for sequence modeling. In Proceedings of NAACL-IILT 2019: Demonstrations, 2019.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pp. 311-318, 2002.
Shrimai Prabhumoye, Yulia Tsvetkov, Ruslan Salakhutdinov, and Alan W Black. Style trans- fer through back-translation. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2018. doi: 10.18653/w1/p18 1080. URL
http: / /xxx doi.
1080
Weizhen Qi, Yu Yan, Yeyun Gong, Dayiheng Liu, Nan Duan, Jiusheng Chen, Ruofei Zhang, and Ming Zhou. Prophetnet: Predicting future n-gram for sequence-to-seqqeene pre-training, 2020.
Lihua Qian, Hao Zhou, Yu Bao, Mingxuan Wang. Lin Qiu, Weinan Zhang, Yong Yu, and Lei Li. Glancing transformer for non-autoregressive neural machine translation. In Proceedings of the 59th Annual Meeting the Association for Computational Linguisties and the I Ith International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 1993-2003 On- line, August 2021. Association for Computational Linguistics. doi: 10. 18653/v1/2022 acl--long. 155. URL https: aaclannntlloyyy org/ 2021 ael22lacclloon long. 155
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 188);9 2019.
Marc' Aurelio Ranzato, Sumit Chopra, Michael Auli, and Wojeiech Zaremba. Sequence level train- ing with recurrent neural networks, 2015.
Yi Ren, Jinglin Liu, Xu Tan, Zhou Zhao, sheng zhao, and Tie- Yan Liu. A study of non- autoregressive model for sequence generation. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020. doi: 10.1865..1 20.18853/v1/2000 1/620////000....mmama. http: doi, htt.:/....../....11////////00000
URL.
Rico Sennrich, Barry Haddow, and Alexandra Birch. Improving neural machine translation models
with monolingual data. Proceedings of the 54th Annual Meeting of the Association for Com- putational Linguistics (Volume 1: Long Papers), 2016a. doi: 10. 10.18663//1/p11-10099 URL
http:://dx www.dooi.oooooo www.ooggg......... http:://............000...11//11 1009
Rico Sennrich, Barry Haddow, and Alexandra Birch. Edinburgh neural machine translation systems for wmt 16. Proceedings of the First Conference on Machine Translation: Volume 2, Shared Task Papers, 2016b. doi: 111111/611111122200 w1 6 2323
URLhtttp /x doi. wrw.orrggloo...o 18..3663/v11
Rico Sennrich, Barry Haddow, and Alexandra Birch. Edinburgh neural machine translation systems for WMT 16. In Proceedings of the First Conference Machine Translation: Volume 2. Shared Task Papers, pp. 371-376, Berlin, Germany, August 2016c. Association for Computational Lin- guistics. doi: 1...8863//1/W11-221 URL wwtphtttp///tt antho oog.oogg//.o.oooooooo....o.oo... 6-2323
13

---

Published as a conference paper at ICLR 2022
Rico Sennrich, Barry Haddow, and Alexandra Birch. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1715-1725, Berlin, Germany, 2016d.
Shikhar Sharma, Layla El Asri, Hannes Schulz, and Jeremie Zumer. Relevance of unsupervised met- rics in task-oriented dialogue for evaluating natural language generation. CoRR, abs/1 706.09799, 2017. http:tt://///
org/ abs /2///1700 1706. C9799
Dinghan Shen, Mingzhi Zheng, Yelong Shen, Yanru Qu, and Weizhu Chen. A simple but tough-to- beat data augmentation approach for natural language understanding and generation, 2020.
Shiqi Shen, Yong Cheng, Zhongjun He, Wei He, Hua Wu, Maosong Sun, and Yang Liu. Minimum risk training for neural machine translation. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1683-1692, Berlin, Germany, August 2016. Association for Computational Linguistics. doi: 10. 10.18853/v1/PH/ 1/P16- 159. URL httpp::/aac.anthooooo..oooo
Connor Shorten and Taghi M Khoshgoftaar. A survey on image data augmentation for deep learning. Journal of Big Data, 6(1):1-48, 2019.
David So, Quoc Le, and Chen Liang. The evolved transformer. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pp. 5877-5886. PMLR, 09-15 Jun 2019. URL https:
mlx. www.pppessssvvee so 9a. html
Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. Learning to summarize with human feedback. Advances in Neural Information Processing Systems, 33:3008-3021, 2020.
Yi Tay, Dara Bahri, Donald Metzler, Da-Cheng Juan, Zhe Zhao, and Che Zheng. Synthesizer: Re- thinking self-attention for transformer models. In International Conference on Machine Learning, pp. 10183-10192. PMLR, 2021.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, f. ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Asso- ciates, Inc., 2017. URL
meurdiiisninnneussss.ee......i.e wtt.cc//pppaaa wttpc//pp/e//21///1eeee.. pdf
Ramakrishna Vedantam, C Lawrence Zitnick, and Devi Parikh. Cider: Consensus-based image description evaluation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4566-4575, 2015.
Dilin Wang, Chengyue Gong, and Qiang Liu. Improving neural language modeling via adversar- ial training. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pp. 6555-6565. PMLR, 09-15 Jun 2019. URL https:: htttps:/prroceed ngs mmlr. press/v9 /ttp/////gg9//...
Xinyi Wang, Hieu Pham, Zihang Dai, and Graham Neubig. SwitchOut: an efficient data aug- mentation algorithm for neural machine translation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 856-861, Brussels, Belgium, October- November 2018. Association for Computational Linguistics. doi: 10.18653/ 1100. URL
/tteps:/acllantholog.... wrg//111.....o-- 1100
Qingsong Wen, Liang Sun, Fan Yang, Xiaomin Song, Jingkun Gao, Xue Wang, and Huan Xu. Time series data augmentation for deep learning: A survey. In Zhi-Hua Zhou (ed.), Proceedings of the Thirtieth International Joint Conference International Joint Conferences on Artificial Intelligence Organization, 8 2021. doi: 10.24963/ ijcai. 2021/31. URL htttt::/t/:/dd http:::///oooooog/////// 211.11.......111 20e.l.202265551 /61
Artificial Intelligence, IJCAI-21, pp. 4653-4660.
Survey Track.
14

---

Published as a conference paper at ICLR 2022
Felix Wu, Angela Fan, Alexei Baevski, Yann Dauphin, and Michael Auli. Pay less attention with lightweight and dynamic convolutions. International Conference on Learning Representations, 2019.
Lijun Wu, Fei Tian, Tao Qin, Jianhuang Lai, and Tie- Yan Liu. A study of reinforcement learning for neural machine translation. Proceedings of the 2018 Conference Empirical Methods in Natural 1397.. URL. http: //x. doi .rgg /
Language Processing, 2018. doi: 10.
18653/v1/a18-1 1397,
Lijun Wu, Shufang Xie, Yingce Xia, Fan Yang, Tao Qin, Jianhuang Lai, and Tie- Yan Liu. Sequence generation with mixed representations. In ICML 2020, July
2020. URL https:://wwtt http://www.icrooofccooooooo con http:///w.....ooo/eeeeeee////eea reggeeeess-eeeeorrrrrrrrreeeeeee
Yingce Xia, Fei Tian, Lijun Wu, Jianxin Lin, Tao Qin, Nenghai Yu, and Tin-Yan Liu. Deliberation networks: Sequence generation beyond one-pass decoding.
I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Gamett (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Asso- neuriiserrppppp. wtc.isscccpppe.. wttccc///paae /cc//ppper/21///llee pdf
ciates, Inc., 2017. URL
Dongling Xiao, Han Zhang, Yukun Li, Yu Sun, Hao Tian, Hua Wu, and Haifeng Wang. Ernie-gen: An enhanced multi-flow pre- trainirinngg and fine-tuning framework for natural language generation. In Christian Bessiere (ed.), Proceedings of the Twenty-Ninth International Joint Conference Artificial Intelligence, IJCAI-20, pp. 3997-4003. International Joint Conferences on Artificial Intelligence Organization, 7 2020. doi: 10.24963//jcai. 20000555. (tLhhttps://// //doi. .trg///o.ooo/ooooooooooooo 10.24963/1jcan 2 2020/553
Main track.
Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Peter Liu. Pegasus: Pre-training with extracted gap-sentences for abstractive summarization. In International Conference on Machine Learning, pp. 11328-11339. PMLR, 2020.
Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. Per- sonalizing dialogue agents: I have dog, do you have pets too? In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp- 2204 2213, Melbourne, Australia, July 2018. Association for Computational Linguistics. doi: 10.18653/v1/PM 1/P1// 1205. / hoyy www.ylorg//..-- 1205
Wen Zhang, Yang Feng, Fandong Meng, Di You, and Qun Liu. Bridging the gap between train- ing and inference for neural machine translation. In Proceedings of the 57th Annual Meet- ing of the Asxociation for Computational Linguistics, pp. 4334-4343, Florence, Italy, July 2019. Association for Computational Linguisties. doi: 10.18653/v1/119111 1426. URL https: //aclanthology. pttp
Li Zhou, Jianfeng Gao, Di Li, and Heung- Yeung Shum. The design and implementation of xiaoice, an empathetic social chatbot. Computational Linguistics, 46(1):53-93, 2020.
A DATA PROCESS AND EVALUATION SCRIPTS
The scripts for data preprocessing and evaluation can be found in the following URLs.
DailyDialogy https///gtthh..ooo////ooo//////
Persona-Chat: htttp///tttaoooo//oooooooaaaaaat projectsspersoneet
IWSLT'14:
Cranslation TTa.s:acccoooorrpeeeeeee.l..._11
14
Cranslation/ ara.
15

---

Published as a conference paper at ICLR 2022
CNN/DM: htte:///thahaannaacrrr/////llll2
sumnarization.md od
Moses
BLEU: http:///s/.alaaao.....e/.//aaaa. scripts/ /ttp://eeeecmemma111ee....p...e
Statistical significance:
master//ccripts/anallyy http:///ecccaaaaa.a.....ooooaaar
p1
files2rouge:
B ABLATION STUDY
In Section 4.4 we mentioned two a schedule formula. Our intuition is that the value should decrease with the traiming progress, while the initial model quality may not be good, which will affect the synthetic data quality. Thus, we believed could be better to have a warm-up stage when the model only learns from real data. Empirically, hyper-parameters do not increase any more after the 50th epoch. That's the reason that we choose 55 in S2. However, compared with fixed value, did not observe improvement. Therefore, kept fixed value in all other experiments.
find that the performance of the model with almost all
Figure contains the plots of these two functions.
1.0
$1
S2
0.8
0.6
0.4
0.2
0
10
20
30
40
50
Epoch
Figure 4: The schedule function.
Besides, due to the limited pages, we put more albation results in Table tables 10
14
C STUDY OF ITERATIVE AUGMENTATION
We show the training curve of IWSLT'14 14 dataset for different iteration round in Figure
D TRAINING CURVE ABOUT THE CONSISTENCY LOSS
We show the training w/ and w/o consistency loss in Figure As it shown in the figure, without consistency loss, the model will converge slower and the performance is worse.
E HUMAN EVALUATION DETAILS
To verify the effectiveness out method on dialogue generation and summarization task, we re- cruited 5 evaluators to judge 100 samples in blind testing. For each response, each evaluator needs to give score from (low quality) to (high qualiy). The score details are shown in Ta- ble 15
16

---

Published as a conference paper at ICLR 2022
Table 10: Ablation study of a on Persona-Chal dataset(dev).
BLEU-1
BLEU-2
BLEU-3
BLEU-4
0.3
18.78
8.57
5.14
3.55
0.4
19.03
8.68
5.21
3.55
0.5
18.99
8.64
5.21
3.55
0.6
18.64
8.53
5.10
3.51
Table 11: Ablation study of softmax temperature IWSLT" 14 dataset.
T
0.5
1
2
4
6
10
BLEU
36.01
36.31
36.28
36.47
36.41
36.23
Table 12: Ablation study of softmax temperature on Persona-Chat
Temperature
BLEU-1
BLEU-2
BLEU-3
BLEU-4
2.0
18.26
8.46
5.11
3.50
4.0
17.92
8.34
5.06
3.49
4.5
19.03
8.68
5.21
3.55
5.0
18.44
8.37
4.98
3.45
6.0
18.27
8.35
5.04
3.48
Table 13:
Ablation on the choice of on dialog(PersonaChat)
D
B1
B2
B3
B4
R-L
Met.
CIDEr
DRL.
19.03
8.68
5.21
3.55
20.20
8.43
21.15
D1s
18.61
8.49
5.08
3.50
19.71
8.43
21.10
Table 14: Ablation on the choice of D summarization.
D
ROUGE-1 ROUGE-2
ROUGE-L
BART+OURS(DK))
44.76
21.46
41.57
BART+OURS(DiS)
44.59
21.42
41.43
Table 15: Human evaluation distribution and average score.
Task
System
3
4
Average Score
Baseline
85.70%
11.68%
1.94%
0.68%
1.175
Dialog
Ours
62.98%
27.90%
6.49%
2.63%
1.487
Baseline 2.27%
14.77%
34.09%
48.86%
3.300
Summarization
Ours
0.00%
9.09%
32.95%
57.95%
3,490
17

---

Published as a conference paper at ICLR 2022
1ITT 12
Inrt Berel 20-2
Bartt
Figure 5: Study of the iterative augmentation on IWSLT.
30
22
Figure 6: BLEU-4 on the validation set during training.
18


Reviews:
[{"review_id": "w1zv9OF-MdJ", "reviewer": "Reviewer_vrEs", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "### Strengths:\nThis paper presents an approach to tackle an important problem in the field of natural language generation.\nIts method is straightforward and well illustrated.\nFurthermore, the paper adheres to scientific standards as the authors release their code.\nIt is also very much appreciated that they illuminate and evaluate their model in different text generation tasks. \nFurthermore, the experimental results demonstrate certain improvements in these tasks.\n\n### Weaknesses:\nMy main concern with this paper is that since the augmented target-side input is produced by the decoder conditioning on the ground truth, this method seems helpful but limited.\nGenerally, given the ground truth, the model produces the target text that is quite similar to its corresponding ground truth, and the discrepancy between them is subtle.\nThus it still cannot narrow the gap of decoding between training time and inference time.\nThe modest improvements in the NMT task and abstract summarization task also indicate the existence of this problem, and cast doubt on its effectiveness.\n\n### Questions for the author(s):\nQ1. As far as I know, in the NMT task, en-->de newstest2014, the best BLEU result for \"OR-NMT\", \"Transformer+Cutoff\", \"Adversarial Training\" are 28.65, 29.1, and 29.52 in their corresponding papers respectively.\nAre there any mistakes?\n\nQ2. The usage of \\alpha on Eq(2) contradicts that on Eq(4)?\n\n", "summary_of_the_paper": "Traditionally, the decoder of Seq2Seq model takes ground truth words of previous steps as input during training, while at inference, its input are those generated tokens starting from scratch.\nThus there exists discrepancy between training and inference.\n\nThis paper presents an approach of data augmentation for the input of the decoder during training. \nSpecifically, the authors still feed the ground truth to the decoder, and then obtain its output vocabulary distributions that are multiplied with word embedding weights to get \"pseudo tokens\".\n\nThe contribution of this work comes from narrowing the gap of decoding procedure between training and testing.", "summary_of_the_review": "Overall, this work tries to tackle the important problem of language generation and proposes a data augmentation method to give soft pseudo decoder input.\nAs what has been mentioned in the weaknesses part of Main Review, there still exist problems to solve and need to prove the effectiveness of such method. Therefore, I'm leaning towards a weak rejection of this paper.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "qpWoKL5GeCN", "reviewer": "Reviewer_ojZ7", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths: \n1. Comprehensive Experiments on three different domains - Dialog, Summarization, and Machine Translation.\n2. Data Augmentation doesn't require access to other resources than the primary model\n\nConcerns/Questions:\n- Renaming Sequence Generation to Sequence-to-Sequence Model (Or Encoder-Decoder) Generation since the domain is restricted to that in this work. Although sequence generation through Language Models might use a similar scheme, this work hasn't been experimented with.\n- Section 2: Comparison with KD: Our method focus on target input, not target output. This part is not very clear. If target output changes, doesn't it also affect target side inputs?\n- Why is training not stabilized with just equation (3). How do you define stabilization of training? Why does consistency loss make the training stable?\n- Number of turns in Tables 1, 2?\n- Is the number of training passes equal for all the methods in Table 3. If not, can it be standardized? It seems like the \"Ours\" model is trained through more passes of the dataset (+ augmentation)\n- Table 4 and 5: Are the results statistically significant? Would you please report those values?\n- Section 4.3: we select the BART to initialize our model, and fine-tune on the CNN/DM -> We fine-tune the BART model on the CNN/DM dataset\n- Section 4.4 [And consequently Table 6-9]: Ablation Study section is not an ablation study, which ideally involves removal of proposed changes [in terms of modeling changes or objective functions]. It seems more like hyperparameter tuning.\n- Section 4.4: How is the value of 55 chosen in $\\alpha$ tuning?\n- Table 10: What happens when \\beta = 0 ? Does the model empirically display stability issues (as mentioned earlier in the motivation for consistency loss)?\n- Study of divergence -> What is the intuition of doing this? Why does KL work better than JS? JS is symmetric, so shouldn't it help?\n- Improper use of \"significantly boost the model\" performance in conclusion when no statistically significant studies have been conducted.\n- Generative tasks require human judgments for proper analysis. Kindly evaluate that.\n- Future Work - \"how to combine this method with unlabeled data is also a significant challenge\" - Why is that so? I believe that the current model also does not utilize any labeled data. Would you please correct me if I am mistaken?\n\n\nTypo/Correction needed:\n- Section 2: Comparison with Iterative Refinement or NAR strategies: Claim about inference cost is incorrect since autoregressive methods are known to be slower than NAR.\n- Equation (3) -> There should be a negative sign in the loss function\n- Capitalize: Rouge -> ROUGE at all places", "summary_of_the_paper": "This paper experiments with a target-side sequence-level data-augmentation scheme for sequence-to-sequence generation tasks. The primary contribution of this work is an algorithm that leverages model-outputs to construct pseudo-target-side tokens (and consequently pseudo-sequences) for augmentation. It is done by incorporating soft-embeddings in the encoder-decoder model, a standard cross-entropy loss function supercharged with a consistency loss objective. One of the critical highlights is that this work doesn't require any external model for performing data augmentation. A comprehensive set of experiments on three sequence generation tasks are conducted to highlight the effectiveness of using such an approach for data augmentation.", "summary_of_the_review": "The work proposes a data-augmentation strategy using soft labels during training instead of hard labels as in scheduled sampling.  One of the key highlights is that this work doesn't require any external model for performing data augmentation. However, the need for consistency loss is unclear, and using this model is not observable. While the paper reports incremental results, the lack of significance testing makes it difficult to comprehend if the results are not by-chance. The ablation study section is not an ablation study but hyper-parameter tuning. Additionally, the lack of experimental details regarding the number of examples seen by each model during training (number of epochs * samples) makes it unclear if the incremental results are due to more extended training or the overall augmentation strategy. \n\nIn its current form, and based on the reasons stated above, I would recommend a weak reject for this paper.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "nNLxxLiO6U5", "reviewer": "Reviewer_iLjj", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n+ The proposed technique is generally applicable to various NLG tasks.\n+ The paper is nicely-structured and very well-written. I like the presentation style of the paper -- a good mixture of formulas for mathematical rigor and textual/graphical explanations of the intuitions behind them.\n+ Experiments span three different generation tasks, showing the general benefit of the proposed augmentation technique, with a detailed ablation study on the sensitivity of some hyper-parameters.\n\nWeaknesses:\n+ The improvements in machine translation and abstractive summarization are marginal compared to other augmentation techniques, but this is not a non-fixable issue, see the first point in my detailed feedback.\n+ I imagine this augmentation method will be computationally expensive, since at every time step $O(|V|d)$ extra additions need to be done ($d$ being the number of word embedding dimensions). It will be useful to have some discussion on that. I also think the number of extra operations can be easily reduced by sparsifying the softmax distribution before doing the embedding interpolation.\n+ I have some minor doubts about the real-world usefulness of the method. For example, for machine translation, isn't this doing a similar job as back-translation?\n\nMore detailed feedback/suggestions:\n+ In the machine translation experiment, the authors compared their method with a series of other data augmentation methods such as WordDrop and SwitchOut, but I think they don't necessarily need to compete with each other -- how about, say, we use SwitchOut on the source side and the proposed augmentation method on the target side? Showing that the proposed method is stackable with other augmentation methods could make this work more impactful.\n+ Continuing on the previous point, it will also be interesting to see if the proposed method is also stackable with back-translation when extra monolingual data is provided, especially since the authors have already considered multi-round enhancement.\n+ Page 3: \"It may also contains\" -> \"It may also contain\"\n+ Page 4: Equation (4) -- should there be an extra summation over $j$?\n+ Page 8: I'm actually not sure why would higher temperature $T$ be helpful under smaller vocabularies. It would be nice to elaborate.", "summary_of_the_paper": "This paper introduces a simple technique for target-side data augmentation. The high-level idea is to generate a \"soft token\" embedding by interpolating the target side word embeddings, with the output distribution generated by an initial model being the interpolation weight (after being adjusted by a temperature parameter $T$). The resulting training loss is in three terms: (1) original cross-entropy loss; (2) cross-entropy loss with the soft token embedding as the input; (3) a consistency loss controlling the distribution divergence of (1) and (2).\n\nResults show that the proposed data augmentation method is very helpful for the dialog generation task, while also being somewhat helpful with neural machine translation for abstractive summarization. Ablation studies show that the weight of (1) cannot be too low, and that the optimal temperature parameter has some negative correlation with the vocabulary size.", "summary_of_the_review": "This paper proposed a simple and novel target-side data augmentation technique that is at least helpful for some natural language generation tasks. I'm going with a weak-accept for now citing some doubts mentioned in the main review, but am happy to update with clarifications from the authors.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "dBaKilQX5_A", "reviewer": "Reviewer_k7SM", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n1. The paper performs comprehensive experiments on three different generation tasks (i.e. dialog generation, machine translation, and abstractive summarization) and demonstrates improvements over baselines.\n2. Their idea of constructing the augmented data can be viewed as a fast approximation of scheduled sampling, which is technically sound.\n3. The paper is easy to follow. \n\nWeaknesses:\n1. They seem to perform ablation studies and choose hyper-parameters on the test sets.\n2. The paper keeps emphasizing that target-side data augmentation has been \"overlooked\" or \"neglected\" in sequence generation throughout the paper, while a lot of papers in this direction have been proposed (e.g. scheduled sampling [1], many RL-based methods [2,3,4], some heuristics [5,6], self-training [7], ...).\n3. The consistency loss is unintuitive and more analyses are required. If the model is trained to output the same targets on the two views, we are already encouraging the consistency between them, so why would an extra consistency loss be helpful here?\n4. The comparisons with baselines are not fair in some cases. The author(s) state that \"for a fair comparison with previous works, we compute the BLEU score by the Moses script \", but the scores obtained by this script depend on the tokenizer, which can be different in different papers and SacreBLEU (https://github.com/mjpost/sacrebleu) is recommended for a truly fair comparison.\n\n\n\n[1] Bengio et al., Scheduled sampling for sequence prediction with recurrent neural networks, NeurIPS 2015.\n\n[2] Norouzi et al., Reward augmented maximum likelihood for neural structured prediction, NeurIPS 2016.\n\n[3] Ranzato et al., Sequence level training with recurrent neural networks, ICLR 2016.\n\n[4] Shen et al., Minimum risk training for neural machine translation, ACL 2016.\n\n[5] Sennrich et al., Edinburgh neural machine translation systems for WMT 16, WMT 2016.\n\n[6] Wang et al., SwitchOut: an Efficient Data Augmentation Algorithm for Neural Machine Translation, EMNLP 2018.\n\n[7] He et al., Revisiting Self-Training for Neural Sequence Generation, ICLR 2020.\n\n", "summary_of_the_paper": "This paper focuses on sequence generation tasks and proposes to perform data augmentation on the target side. During training, given an input-output pair ($x$, $y$), their model first uses teacher-forcing to get the output probability $p_y$ and then gets an augmented pair ($x$, $\\tilde{y}$) by using \"mixup\" to mix the embeddings of $y$ and $p_y$. They also propose to encourage the consistency of predictions between ($x$, $y$) and ($x$, $\\tilde{y}$) by minimizing the KL divergence of their output probabilities.\n\nThey experiment on three sequence generation tasks and demonstrate improvements over baselines.", "summary_of_the_review": "While I think their idea of augmenting the dataset is interesting and it does not introduce any inference burden, based on the weaknesses I mentioned above, I'm leaning towards a rejection of the paper.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents a method for target side data augmentation for sequence to sequence models.  The authors of the paper use a relatively straightforward method to generate pseudo tokens that are used for enhanced training.  The authors present results on dialog generation, MT and summarization where automatic metrics show improvements.  For really robust results, I would have preferred to see more human evaluations since BLEU and ROUGE are metrics that the NLP community is moving away from.  Overall, the majority of the reviewers are happy with the paper and there is significant back and forth between the reviewers and authors that have improved the paper;  I think the authors went to significant lengths to allay all concerns from the reviewers and the paper should be accepted.
Decision: Accept (Poster)