Reweighting Augmented Samples by Minimizing the Maximal Expected Loss
Authors: Mingyang Yi
Lu Hou
Lifeng Shang
Xin Jiang
Qun Liu
Zhi-Ming Ma
Venue: ICLR 2021 Poster (2021)
Keywords: data augmentation
sample reweighting

Abstract: Data augmentation is an effective technique to improve the generalization of deep neural networks. However, previous data augmentation methods usually treat the augmented samples equally without considering their individual impacts on the model. To address this, for the augmented samples from the same training example, we propose to assign different weights to them. We construct the maximal expected loss which is the supremum over any reweighted loss on augmented samples. Inspired by adversarial training, we minimize this maximal expected loss (MMEL) and obtain a simple and interpretable closed-form solution: more attention should be paid to augmented samples with large loss values (i.e., harder examples). Minimizing this maximal expected loss enables the model to perform well under any reweighting strategy. The proposed method can generally be applied on top of any data augmentation methods. Experiments are conducted on both natural language understanding tasks with token-level data augmentation, and image classification tasks with commonly-used image augmentation techniques like random crop and horizontal flip. Empirical results show that the proposed method improves the generalization performance of the model.

Published as a conference paper at ICLR 2021
REWEIGHTING AUGMENTED SAMPLES BY MINIMIZ- ING THE MAXIMAL EXPECTED LOss
Mingyang 2 Hou?, Lifeng Shang", Xin Jiang", Qun Liu", Zhi-Ming Ma! University of Chinese Academy of Sciences yi mimigggnngg7eee 7@mails. ucas. edu. cn 2 Academy of Mathematics and Systems Science, Chinese Academy of Sciences ma mazmaamn anm ac.cs cn
Huawei Noah's Ark Lab (houlu3, shang. 1 feng, Jiang. Xin, qun. ltwllluuuuuueeee hhawwii.com
ABSTRACT
Data augmentation is an effective technique to improve the generalization of deep neural networks. However, previous data augmentation methods usually treat the augmented samples equally without considering their individual impacts on the model. To address this, for the augmented samples from the same training example, we propose assign different weights to them. We construct the maximal expected loss which is the supremum over any reweighted loss augmented samples. Inspired by adversarial training, we minimize this maximal expected loss (MMEL) and obtain simple and interpretable closed-form solution: more attention should be paid to augmented samples with large loss values (i.e., harder examples). Minimizing this maximal expected loss enables the model perform well under any reweighting strategy. The proposed method can generally be applied top of any data augmentation methods. Experiments natural language understanding tasks with token-level data augmentation, and image classification tasks with commonly-used image augmentation techniques like random crop and horizontal flip. Empirical results show that the proposed method improves the generalization performance the model.
conducted both
INTRODUCTION
Deep neural networks have achieved state-of-the-art results in various tasks in natural language processing (NLP) tasks (Sutskever al., 2014 Vaswani al., 2017: Devlin et al. 2019) and computer vision (CV) tasks (He et al.] 2016 Goodfellow et al. al.] 2016). One approach to improve the generalization performance of deep neural networks is data augmentation (Xie al. 2019 Jiao et al. 2019: Cheng et al.] 2019: 2020). However, there are some problems we directly incorporate these augmented samples into the training set. Minimizing the average loss on all these samples means treating them equally, without considering their different implicit impacts on the loss.
To address this, we propose to minimize reweighted loss on these augmented samples make the model utilize them in a cleverer way. Example reweighting has previously been explored extensively in curriculum learning (Bengio
al. 22099] Jiang et al.) 2014). boosting algorithms (Freund & Schapire 1999), focal loss (Lin et al. 2017) and importance sampling (Csiba & Richtárik] 2018)). However, none of them focus on the reweighting of augmented samples instead the original training samples. A recent work (Jiang et al.) 2020a) also assigns different weights on augmented samples. But weights in their model are predicted by a mentor network while we obtain the weights from the closed-form solution by minimizing the maximal expected loss (MMEL). In addition, they focus on image samples with noisy labels, while our method can generally be applied to also textual data as well as image data. Tran et al. (2077) propose to minimize the loss on the augmented samples under the framework of Aopectation-Maxxiizaan algorithm. But they mainly focus on the generation of augmented samples.
"This work is done when Mingyang Yi is intern at Huawei Noah's 's Ark Lab.

---

Published as a conference paper at ICLR 2021
Unfortunately. in practise there is no way to directly access the optimal reweighting strategy. Thus, inspired by adversarial training (Madry et al. 2018). we propose to minimize the maximal expected loss (MMEL) augmented samples from the same training example. Since the maximal expected loss is the supremum over any possible reweighting strategy on augmented samples' losses, minimiz- ing this supremum makes the model perform well under any reweighting strategy. More importantly, we derive a closed-form solution of the weights, where augmented samples with larger training losses have larger weights. Intuitively, MMEL allows the model keep focusing on augmented samples that are harder to train.
The procedure of our method is summarized as follows. We first generate the augmented samples with commonly-usee data augmentation technique, e.g., lexical substitution for textual input (Jao et al. 2019), random crop and horizontal flip for image data (Krizhevsky et al., 2012). Then we explicitly derive the closed-form solution of the weights on each of the augmented samples. After that, we update the model parameters with respect to the reweighted loss. The proposed method can generally be applied above any data augmentation methods in various domains like natural language processing and computer vision. Empirical results on both natural language understanding tasks and image classification tasks show that the proposed reweighting strategy consistently outperforms the counterpart of without using it, as well as other reweighting strategies like uniform reweighting.
2 RELATED WORK
Data augmentation. Data augmentation is proven to be an effective technique to improve the generalization ability of various tasks, e.g., natural language processing (Xie et al. 2019 Zhu et al. 2020 Jiao et al.] 2019), computer vision (Krizhevsky et al. 2014), and speech recognition (Prr et al. 2019). For image data, baseline augmentation methods like random crop. flip, scaling, and color augmentation (Krizhevsky et al., 2012) have been widely used. Other heuristic data augmentation techniques like Cutout (De & Taylor 2017) which masks image patches and Mixup ZZhagg et al. 2018) which combines pairs of examples and their labels, are later proposed. Automatically searching for augmentation policies (Cubuk et al. 2018 Lim et al. 2019) have recently proposed to improve the performance further. For textual data, Zhhng et al. (2015); Wei & Zou| (2019) and
Wang (2015) (205) respectively use lexical substitution based on the embedding space. Jiao et al. (2019);
Cheng et al. )2019); Kumar al. (2020) generate augmented samples with pre-trained language
model. Some other techniques like back translation (Xic ct al. 2019). random noise injection (Xic
et al. 2017) and data mixup (Guo et al. 2019 Cheng et al. 2020) are also proven to be useful.
Adversarial training. Adversarial learning is used to enhance the robustness of model (Madry cet al., 20188) which dynamically constructs the augmented adversarial samples by projected gradient descent across training. Although adversarial training hurts the generalization of model on the task of image classification (Raghunathan et al. 2019), it is shown that adversarial training can be used as data augmentation to help generalization in neural machine translation (Cheng et al.) 2019 (020) and natural language understanding (Zhu et al. 2020 Jiang et al., 2020b). Our proposed method differs from adversarial training in that we adversarially decide the weight on each augmented sample, while traditional adversarial training adversarially generates augmented input samples.
In (Bchpour et al.) 2019), adversarial learning is used as data augmentation in object detection. The adversarial samples (i.c., bounding boxes that are maximally different from the ground truth) are reweighted to form the underlying annotation distribution. However, besides the difference in the model and task, their training objective and the resultant solution are also different from ours.
Sample reweighting. Minimizing a reweighted loss on training samples has been widely explored in literature. Curriculum learning (Bengio et al.) 2009 Diang et al. 2014) feeds first easier and then harder data into the model to accelerate training. Zhao & Zhang 2014); Needell et al. (2014); Csiba & Richtárik (2018): Katharopoulos & Fleuret (2018) use importance sampling to reduce the variance of stochastic gradients to achieve faster convergence rate. Boosting algorithms (Freund & Schapire, 1999) choose harder examples to train subsequent classifiers. Similarly, hard example mining (Malisiewicz et al.] 2011) downsamples the majority class and exploits the most difficult examples. Focal loss (Lin et al. [217]] Goyal & He. 2018) focuses on harder examples by reshaping the standard cross-entropy loss in object detection. Ren et al. (211)): Jiang et al. (2018); (Shu et al.
(2019) use meta-learning method to reweight examples to handle the noisy label problem. Unlike all
2

---

Published as a conference paper at ICLR 2021
these existing methods, in this work, we reweight the augmented samples' losses instead of training samples.
MINIMIZE THE MAXIMAL EXPECTED Loss
In this section, we derive our reweighting strategy on augmented samples from the perspective of maximal expected loss. We first give a derivation of the closed- solution of the weights on augmented samples. Then we describe two kinds of loss under this formulation. Finally, we give the implementation details using the natural language understanding task as an example.
3.1
WHY MAXIMAL EXPECTED LOSS
Consider a classification task with training samples. For the i-th training sample w,, its label is denoted Ve, Let fo(-) be the model with parameter which outputs the classification probabilities. f(.,)) denotes the loss function, e.g. the cross-entropy loss between outputs fo fo(æ;) and the ground- truth label Ya,-- Given original training sample æ; the set of augmented samples generated by some method is B(:)). Without loss of generality, we assume æ; E B(x,). The conventional training objective is to minimize the loss on every augmented sample in B(x,) as
min N
((fozz),
B(ax)]
((Bm))
where ys is the label of x E B(x;), and can be different with Vz.. [B(x;)] is the number of augmented samples in B(x;), which assumed be finite.
In equation I for each given e;, the weights on its augmented samples are the same (i.e., ((x,)))))))))))))) However, different samples have different implicit impacts on the loss, and we can assign different weights on them to facilitate training. Note that computing the weighted sum of losses of each augmented sample B (a:)) can be viewed as taking expectation of loss on augmented samples E (xx:) under certain distribution. When the augmented samples generated from the same training sample are drawn from uniform distribution, the loss in equation [[ can be rewritten as
N
min Ro )- min
Σ EZ-to(in:) [e(fo(z), y=)] - (P x:) || PP xx))]]
where the Kullback-Leibler (KL) divergence (I | x;)) || Pr x;)) equals zero. Here Pr((1 | æ;) denotes the uniform distribution on B(æ;). When the augmented samples are drawn from a more general distribution (- [instead of the uniform distribution, we can generalize Pu here to some other conditional distribution PB.
min (P min 0
Σ
Ba-rnt [e(o(z) v=)] XpKL x.)) Pu [x.))]
(3)
N
Remark 1. When FB(- x.) reduces to the uniform distribution Pu(- xx)) for any 2; since (L(Pu xx) Pu(- xx)) = 0, the objective in equation 3 reduces to the one in equation ①I.
The KL divergence term in equation is used as regularizer to encourage P& close to Py (see Remark 2 From equation (3). the conditional distribution Pg determines the weights of each augmented sample in (xx)). There may exist optimal formulation of Pa in some regime, e.g. corresponding to the optimal generalization ability of model. Unfortunately, we can not explicitly characterize such unknown optimal Ppg. To address this, we borrow the idea from adversarial training (Madry et al. (018)) and minimize the maximal reweighted loss on augmented samples. Then, the model guaranteed to perform well under any reweighting strategy, including the underlying optimal one. Specifically, let the conditional distribution PP be TP = argsupr, Ra (PPB). Our objective is to minimize the following reweighted loss
min (1%) = min sup (IP
(4)
Pg
The following Remark discusses about the KL divergence term in equation 3
In the following, simplify Ps IP if there is no obfuscation.

---

Published as a conference paper at ICLR 2021
Remark 2. Since we take supremum over PB in equation 4 the regularizer KL(FB || Pu) encourages PB to be close to Py because it reaches the minimal value zero when PB = Pt. Thus the regularizer controls the diversity among the augmented samples by constraining the discrepancy between b and uniform distribution Pu, egg., a larger Xp promotes a larger diversity among the augmented samples.
The following Theorem gives the explicit formulation of RF(F)))
Theorem 1. Let (2 and (P()) be defined equation and 44, then we have
Pa(z m.){((((2)) ApPā(z a.) log (P((2))|PP(2 m:)))
(Po)
Σ
Σ [(E((
(5)
where
Xp ([0(2)))zz))
I
exp
Softmax; ( [(
Po(z wi) =
(6)
ELED(e.) exp ((f(())).
Ap
where Softmaxx Ae ((fo UB(a.))) represents the output probability of z for vector
Xp ((f(()) (2.)))))))..- Uz.). + d (fo (=IB(=)))) B(B(a))).
Remark 3. we ignore the KI. divergence term in equation due to the equivalence of minimizing cross-entropy loss and MLE loss (Martens) 2019) the proposed MMEL also falls into the generalized pnnectatioonMaxiiiaaait
(GEM) framework (Dempster al. 1977y. Specifically, given training example, the augmented samples of it can be viewed as latent variable, and any reweighting on these augmented samples corresponds to a specific conditional distribution of these augmented samples given the training sample. In the expectation step (E-step), we explicitly derive the closed-form solution of the weights each of these augmented samples according (6) In the maximization step, since there is no analytical solution for deep neural networks, following (Tran el al.) [017]] we update the model parameters with respect to the reweighted loss by one step of gradient descent.
The proof of this theorem can be found in Appendix A From Theorem I the loss of it decides the weight on each augmented sample E Ba and the weight is normalized by Softmax over all augmented samples in Bzg- The reweighting strategy allows more attention paid to augmented samples with higher loss values. The strategy is similar those (Lin et al.) 2017 Zhao & ZZang 2014) but they apply it on training samples.
3.2 Two TYPES OF LOss
For augmented sample z E B(x;), instead of computing the discrepancy between the output probability fo(z) and the hard label y/ in equation (5), can also compute the discrepancy between fa(z) and the "soft" probability fo(x;) in the absence of ground-trutt label on augmented samples as in (iie et 2019). the following, We use superscript "hard" for the loss in equation (5) as
Rhard (Pt
Po(2 m)){(f(())) v=)) - ApPo(2 x;) log (IB(a))) mm)),,
(7)
BEB(w.)
to distinguish with the following objective which uses the "soft probability":
R (Pt, xx) ((fo(x)), V + Ar
(Pa(2 a m)){(f(())) fo(x)))
(8)
>x æ.) log (IB(x.)) ()P x.)).
The two terms in 1ezoft(F's æx.) respectively correspond to the loss on original training samples and the reweighted loss on the augmented samples. The reweighted loss promotes small discrepancy between the augmented samples and the original training sample. Xr > O is the coefficient used to balance the two loss terms, and Ln (z æ;) is defined similar to (6) as
exp [(fo(z),
P5 xx) =
(9)
Σ Eze (=:); exp J/2 ((f(())) fo(m))
4

---

Published as a conference paper at ICLR 2021
R (P..))
KP
x Ph
xper
fe(x.)
(a
fx())
folem-))
x1
(a) MMEL-II
(b) MMEL-S.
Figure 1: MMEL with two types of losses. Figure Tab the hard loss 7 with probability computed
using 6 while Figure (lb) is the soft loss 8 with the probabilities computed using
Algorithm 1 Minimize the Maximal Expected Loss (MMEL)
Input: Training set {(01 Ve, ..... (IN New)) }}. batch size S, learning rate number of training iterations T, Ro equals Raaad or Reyoft
generate augmented samples 2: Generating B(x;)) using some data augmentation method.
1: for in 21, .N3 do
3: end for
4: fo T do
> minimize the maximal expected loss 5: Randomly sample mini-batch S {(21, yma, ..... (Zig.ya,g) from training set.
6: Fetch the augmented samples B ((xi)) (B(rig)).
7: Compute 17 according to (6) (9).
8: Update model parameters 0.+1 = 0e 3 Exes VoRo(P x).
9: end for
The two losses are shown in Figure of reweighted training objectives.
T Summing over all the training samples, we get the two kinds
Remark 4. The proposed MMEL-S tries to reduce the discrepancy between fo(z) and fo(xx;) ) for (xx.). However, if the prediction fe(x:) is inaccurate, such misleading supervision for may lead to the degraded performance of MMEL-S. More details are in Appendix B
3.3 EXAMPLE: MMEL IMPLEMENTATION ON NATURAL LANGUAGE UNDERSTANDING TASKS
In this section, we elaborate on implementing the proposed method using textual data in natural language understanding tasks example. Our method is separated into two phases. In the first phase, we generate augmented samples. Then in the second phase, with these augmented samples, we update the model parameters under these augmented samples with respect to the hard reweighted loss (7) or the soft counterpart (8). The generation and training procedure can be decoupled. and the augmented samples are offline generated in the first phase by only once. On the other hand, in the second phase, since we have the explicit solution of weights on augmented samples and the multiple forward and backward passes on these augmented samples can be computed in parallel, the whole training time is similar to the regular training counterpart for an appropriate number of augmented samples. The whole training process shown Algorithm ]
Generation of Textual Augmented Data. Various methods have been proposed to generate aug- mented samples for textual data. Recently, large-scale pre-trained language models like BERT (Devlin) ctt al. 2019) and GPT-2 (Radford et al., 2019) learn contextualized representations and have been used widely in generating high-quality augmented sentences (Jiao et al.) 2019 Kumar al. 2020). In this paper, we use pre-trained BERT trained from masked language modeling to generate augmented samples. For each original input sentence, randomly mask k tokens. Then do forward propagation of the BERT to predict the tokens in those masked positions by greedy search. Details can be found in Algorithm 2 in Appendix C
5

---

Published as a conference paper at ICLR 2021
Mismatching Label. For Rbpad in equation (7). the loss term ((fo(z),)) yz) augmenteed sample x E B(x;) for some 2; relies on its label 1/2- Unlike image data, where conventional augmentation methods like random crop and horizontal flip of an image do not change its label, substituting even one word in a sentence can drastically change its meaning. For instance, suppose the original sentence is "She is my daughter", and the word "She" is masked. The top words predicted by the pre-trained BERT are "This, She, That, It, He". Apparently, for the task of linguistic acceptability task, replacing "She" with "He" can change the label from linguistically "acceptable" textual input, for the term {[f((z), yz) in hard loss (7) instead of directly setting yz as yv. (Zhu et al. 2020), we replace y/= with the output probability of a trained teacher model. On the other hand, for the soft loss in equation (8), if augmented sample E (ax;) is predicted to a different class from a;; by the teacher model, unreasonable to still minimize the discrepancy between [0(2) and (a,)). In this case, we replace (ax, in the loss term AT E Exe B(a)))a); Po xx))(fo(z)) So (a())) with the output probability from the teacher model.
non--accettable,, Thus for
EXPERIMENTS
In this section, we evaluate the efficacy of the proposed MMEL algorithm with both hard loss (MMEL-H) and soft loss (MMEL-S). Experiments are conducted on both the image classification tasks CIFAR- 10 and CIFAR-100 (Krizhevsky et al.) (2014) with the ResNet Model (He et al., 2016), and the General Language Understanding Evaluation (GLUE) tasks (Wang et al. 2019) with the BERT model (Devlin al. 2019).
4.1 EXPERIMENTS ON IMAGE CLASSIFICATION TASKS.
Data. CIFAR Krizhevsky et al. 2014) is a benchmark dataset for image classification. We use both CIFAR-10 10 and CIFAR-100 in experiments, which are colorful images with 50000 training samples and 10000 validation samples, but from 10 and 100 object classes, respectively.
Setup. The model we used is ResNet (He et al. 2016) with different depths. We use random crop and horizontal flip (Krizhevsky et al. (012)) to augment the original training images. Since these operations do not change the augmented sample label, we directly adopt the original training sample label for all its augmented samples. Following (He et al. 2016),, we use the SGD with momentum optimizer to train each model for 200 epochs. The learning rate starts from 0. and decays by a factor of 0.2 epochs 60, 120 and 160. The batch size is 128, and weight decay is 5e-4. For each æ; [B(x.)] = 10. The Ap of the KL regularization coefficient is 1.0 for both MMEL-H and MMEL-S. The \T in equation (8) for MMEL-S selected from (0.5, 1.0, 2.0].
We compare our proposed MMEL with conventional training with data augmentation (abbreviated as "Baseline(DA))) under the same number of epochs. Though MMEL can be computed efficiently in parallel, the proposed MMEL encounters [Bx;)) I = 10 times more training data. For fair comparison, we also compare with two other baselines that also use 10 times more data: (i) naive training with data augmentation but with 10 times more training epochs compared with MMEL (abbreviated as "Baseline(DA+Lonngg)
In this case, the learning rate accordingly decays at epochs 600, 1200 and 1600; (ii) training with data augmentation under the framework of MMEL but with uniform weights on the augmented samples (abbreviated as "BaselineDA+UNN))),
Main Results. The results are shown in Table I As can be seen, for both CIPAR-10 and CIFAR-100, MMEL-H and MMEL-S significantly outperform the Baseline(DA), with over 0.5 points higher accuracy on all four architectures. Compared to Baseline(DA+Longg) the proposed MMEL-H and MMEL-S also have comparable or better performance, while being much more efficient in training. This is because our backward pass only computes the gradient of the weighted loss instead of the separate loss of each example. Compared to Baseline(DA+UND) which has the same computational cost as MMEL-H and MMEL-S, the proposed methods also have better performance. This indicates the efficacy of the proposed maximal expected loss based reweighting strategy.
We further evaluate the proposed method on larege-scale dataset ImageNet (Deng et al. 2009). The detailed results are in Appendix B

---

Published as a conference paper at ICLR 2021
Table 1: Performance of ResNet on CIFAR-10 and CIFAR-100, The time is the training time measured on single NVIDIA V100 GPU. The results of five independent runs with "mean (+sstd) are reported, expected for "Baseline(DA + Long)" which is slow training.
Bascline(DA)
GaselmeeDAUUN
MMEL-H
MMEEL-S
dalaset
Model
RCC
time
aCc
lime
time
aCc
time
RCC
time
92.3110
0.21
93.10
10.10110.188
2.96
CTPAR-10
ResNetd4
ResNer32
20.46)+0.21)
(0)
0.7h
0.8h
94,11
94.43
7:2h
9.36
94, 94.30; 18)
94.11((0.33)
4.3h
5.7b
94.31(-0.07)
-4.70K-0.14)
5.7h
4.3h
94.48)
4.3h
5.7h
Re
93.96(40.00)
1.1h
94,12
94.62/+0.0
7.06
94.85(
0.155
04.64(
641
7.06
ResNer30
68.95(20.56)
0.2h
69.45
6.75
(0.09
2.9h
70.01 0.07)
2.9h
70.001
20.07)
2.9h
ResNet32
70.66(10.16)
0.7h
71.98
7.2h
71.59;1:..00
4.36
02.57(00.07)
4.3h
72.57(1:0.200 20)
4.36
C1PAR-100
ResNer44
71.43(+0330)
0.8h
72,83
8.3h
72. 30
5.7h
73.1%
00.313
5.7h
72.89
5.7h
ResNet56
72.22(40.26)
1.1h
73.09
10.60
73.44(40.13)
7.06
-4.20
7.0h
73.89(30.15)
7.06
Varying the Number of Augmented Samples. One hyperparumeter of the proposed method is the number of augmented samples [B(x.)]. In T we evaluate the effect of [axx)) on the CIFAR dataset. We vary [B(x))] in {2,5. 5. 10, 20} for both MMEL-H and MMEL-S with other settings unchanged. As can be seen, the performance of MMEL improves with more augmented samples for small [xx)). However, the performance gain begins to saturate when [Bx))) reaches or 10 for some cases. Since a larger [Bx)) also brings more training cost, we should choose a proper number of augmented samples rather than continually increasing it.
Table 2: Performance of MMEL on CIFAR-1 and CIFAR- with ResNet with varying |Bal Here "MMEL-*-A means training with MMEL-* loss with [B(x)) = k. The results are averaged over five independent runs with "mean(=std)" reported.
dataset
Model
Da)cline(DN)
MM
CCFAR-10
RoNNer12
93.02=0.10)
9988--0.16
14.88-00122
09.77:0013)
04.20;-0.18)
04.11:00.13
94.35:40.07)
14.00:00144
04.10-0.00)
04.48(+0.06)
01.99/+0.05)
94.30
01.8%
RoNic5
93
04.00:0000)
04.48:00.05
04.88:0000)
94.56:20.15
14.5
04.64(20.03)
95
04.60110
5605
10000:00361
60,38 20.55000111
18
C1P38-100
ReoNe:2
RosNer46
21 21.43-00300
2066620.16)
72. 12.08:00000
11.50:10.00
71337200.30
12.42(90.24)
71.73:20.1 12411200118
13.00:00.03
72.92(10.15
72.51110.00)
73 (3.8:00.30
72
22.57(10.20)
72.25(20.11)
73.23(00.18)
72.77610 15)
12.00110..1 12)
Ro
-2222-00366
78:12:10.66
73.35+00300
73.95(+0.04)
73.47(10.14)
74.20(10.24)
73.89(+.115)
74.10(+0.05)
73.53++011))
4.2 RESULTS ON NATURAL LANGUAGE UNDERSTANDING TASKS
Data. GLUE is a benchmark containing various natural language understanding tasks, including tex- tual entailment (RTE and MNLI), question answering (QNLI), similarity and paraphrase (MRPC, QQP, STS-B) sentiment analysis (SST-2) 2) and linguistic acceptability (CoLA), Among them, STS B regression task, CoLA and SST-2 are single sentence classification tasks, while the rest sentence-pair classification tasks. Following (Devlin et al.) 2019), for the development set, we report Spearman correlation for STS-B Matthews correlation for CoLA and accuracy for the other tasks. For the test set for QQP and MRPC, we report "F1".
Setup. The backbone model is BERT (Devlin et al. 2019). We use the method in Section 3.3 generate augmented samples. For the problem of mismatching label described in Section 3.3 we use a BERTBASE model fine-tuned on the downstream task as teacher model to predict the label of each generated sample z in Bxx,). For each æ,, [B(x))] = 5. The fraction of masked tokens for each sentence is 0.4. The Xp of the KIL regularzzation coefficient is 1.0 for both MMEL--H and MMEL-S. The X in equation (8) for MMEL-S 1.0. The other detailed hyperparameters training
can be found in Appendix [
The derivation of MMEL in Section is based on the classification task, while STS -BSBBR a regression task. Hence, we generalize our loss function accordingly for regression tasks as follows. For the hard loss in equation 7. we directly replace y/ E R with the prediction of teacher model on z. For the soft loss (8) for each entry of fo(x,) in loss term XT E Pa(x x; M.. (=)( fo we replace it with the prediction of teacher model if the difference between them is larger than 0.5.
We compare with three baselines. However, we change the first baseline to naive training without data augmentation (abbreviated "Baseline") since data augmentation is not used by default in NLP tasks. The other two baselines are similar to those in Section [4T] (i) "Baseline(DA+LLong" which fine-tunes BERT with data augmentation with the same batch size; and
Similar to Section 4.1
7

---

Published as a conference paper at ICLR 2021
87.5
85.0
50-
88
+8.0
Speemmnn
77.5
MHELL
HATD
7/3
Esochs
Epocns
Epocis
Epocts
(a) CoLA.
(b) SST-2.
(c) MRPC.
(d) STS-B.
93-
90-
Rouats witieri
38-
Hart, a
57
0.0
2.5
5.0
75
10.
255
5.0
7.5
10.0
Toocha
Epochs
Epocts
Epects
(e) QQP.
MNLI.
QNLI.
(h) RTE.
Figure Development set results on BERTBASE BASE model with different loss functions.
(ii)" "DA-Ulin which fine-tunes BERT with augmented samples by using average loss. We also compare with another recent data augmentation technique SMART (Tian al. 2020b).
Table 3: Development and test sets results on the BERT model. The training time is measured on single NVIDIA V100 GPU. The results Baseline, BDseline MMEL-H and MMEL-S are obtained by five independent runs with "mean( std) reported.
Method
CoLA
8.Nc
S21-2
676
HBUIC
1.76
7%
DGP
164k
70%
CHL2
1096
2.%G
Ang
Time
Buscluse
39.1120011
95. 15
37.10 30.000056
PL. PLTEER
2133
61.5
91.3
$1.1
84.3
DA-UUN)
61.3)5075)
93. 177077-777)
67.K-0063)
900
71.81+1003
SMART
91.5
856666
8400
MIMEL-B
62.cc30.55
93, (3.004).14)
07.70-0.0
00.4:014)
853:10
92.2610.10
22.3(10.55
84.5116.1
31.8h
MMBLLSS
613r30.153
MK. 0.7)
V2.4--0111 135
71 71.5040.24
366-30.11)
37.4h
Basclue
$1,6100733
93.311:011
06.0
71 (13:2)33))
300
67.4021100)
79.6430000
21.5h
BaseimeehAAee ameg
50.0
91.3
31.5
$4.839
8h.1
BasclagDAAUNN
52.4011.50)
92.362:052)
87.7
05800070)
00:61:0433
63.8111.76)
79:70200500
MMIFL-11
MMEEL
52.5650.55
93.5:00110
88.3:00115)
72.10501
350::0025842:50001
91.4:20333)
6939110.54
40.3640.100
Main Results. The development and test set results on the GLUE benchmark are shown in Table
3 The development set results for the BERT baseline are from our re-implementation, which is comparable or better than the reported results in the original paper (Devlin et al.) 2019). The results for SMART taken from (Jiang et al. 2020bb, and there are test results in (Jiang al. 2020b). As can be seen, data augmentation significantly improves the generalization of GLUE tasks. Compared to the baseline without data augmentation (Baseline), MMEL-H or MMEL-S consistently achieves better performance, especially on small datasets like CoLA and RTE. Similar to the observation in the image classification task in Section 4.1 the proposed MMEL-H and MMEL-S are more efficient and have better performance than BaselineDA+Lonng) MMEL-H and MMEL-S also outperform BaselineDAA+UND, indicating the superiority of using the proposed reweighting strategy. In addition, our proposed method also beats SMART in both accuracy and efficiency because they use PGD-k (Madry et al.) 2018) to construct adversarial augmented samples which requires nearly A times more training cost. Figure 2 shows the development set accuracy across over the training procedure. As can be seen, training with MMEL-H or MMEL-S converges faster and has better accuracy except SST-2 and RTE where the performance is similar.
Effeet of Predicted Labels. For the augmented samples from same origin, we use a fine-tuned task-specific BERTBASE teacher model to predict their labels as mentioned in Section 3.3 to handle the problem of mismatching label. In Table we show the comparison between using the label of the original sample and using predicted labels. As can be seen, using the predicted label significantly improves the performance. By comparing with the results in Table using the label of the original sample even hurts the performance.
8

---

Published as a conference paper at ICLR 2021
Table 4: Effect of using the predicted label. Development set results are reported.
Method
Label
ColAA
SST-2
MRPC
STS-B B
QOP
MNLI -mm mm
QNLI
RTE
Avg
MMEL-H
Original
48.8
91.5
79.2
80.3
88.6
80.4/79.8
88.8
65.3
78.1
MMEL-H
Predicted
62.8
93.3
87.5
90.4
91.6
85.4/85.6
92.1
72.2
84.5
MMEL-S
Original
56.6
91.7
85.8
81.6
90.0
81.9/81.3
89.9
61.0
80.0
MMEL-S
Predicted
61.7
93.8
89.2
90.2
91.6
85.0/85.5
92.5
73.3
84.7
5
CONCLUSION
In this work, propose minimize reweighted loss over the augmented samples which directly considers their implicit impacts on the loss. Since we can not access the optimal reweighting strategy, we propose to minimize the supremum of the loss under all reweighting strategies, and give closed- form solution of the optimal weights. Our method can be applied on top of any data augmentation methods. Experiments on both image classification tasks and natural language understanding tasks show that the proposed method improves the generalization performance of the model, while being efficient in training.
REFERENCES
Behpour, K. Kitani, and B. Ziebart. Ada: Adversarial data augmentation for object detection. In IEEE Winter Conference on Applications of Computer Vision, 2019.
Y. Bengio, J. Louradour, R. Collobert, and J. Weston, Curriculum learning. In International Conference on Machine Learning, 2009.
Y. Cheng, L. Jiang, and W. Macherey. Robust neural machine translation with doubly adversarial inputs. In Annual Meeting of the Association for Computational Linguistics, 2019.
Y. Cheng, L.. Jiang, W. Macherey, and J. Fisenstein. Advaug: Robust adversarial augmentation for neural machine translation. In Annual Conference of the Association for Computational Linguistics, 2020.
D. Csiba and P. Richtárik. Importance sampling for minibatches. The Journal of Machine Learning Research, 19(1):962-982 2018.
E. D. Cubuk, B. Zoph, D. Mane, V. Vasudevan, and Q V. Le. Autoaugment: Learning augmentation policies from data. Preprint Xiv: Seti:1805.000 805.805..90999 005011911 2018.
A. P. Dempster, N. M. Laird, and D. B. Rubin. Maximum likelihood from incomplete data via the em algorithm. Journal of the Royal Statistical Society: Series B (Methodologicul), 39(1):1-22, 1977.
J. Deng, W. Dong, R. Socher, L. Li, K. Li, and L. Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Putlern Recognition, 2009.
Devlin, M. Chang, K. Lee, and K. Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In North American Chapter of the Association for Computational Linguistics, 2019.
T. DeVries and G. W. Taylor. Improved regularization of convolutional neural networks with cutout. Preprint arXiv: 1708 008580 2017.
Y. Freund and R. Schapire. A short introduction to boosting. Journal-lapanese Society For Artificial Intelligence, 14(771-780): 1999.
I. Goodfellow, Y. Bengio, and A. Courville. Deep learning, volume 2016.
P. Goyal and K. He. Focal loss for dense object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39:2999-3007,, 2018.
H. Guo, Mao, and R. Zhang. Augmenting data with mixup for sentence classification: An empirical study. Preprint 1905 2019.

---

Published as a conference paper at ICLR 2021
K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition, 2016.
L. Jiang, D. Meng, S. Yu, Z. Lan, S. Shan, and A. Hauptmann. Self-pacer learning with diversity. In Advances in Neural Information Processing Systems, 2014.
L. Jiang, Z. Zhou, T. Leung. L. Li, and Li. Mentornet: Learning data-driven curriculum for very deep neural networks on corrupted labels. In International Conference on Machine Learning, 2018.
I. Jiang, Huang, M. Liu, and Yang. Beyond synthetic noise: Deep learning on controlled noisy labels. In International Conference Machine Learning, 2020a.
P. Jiang, Hand He, W. Chen, X. Liu, J. Gao, and T. Zhao. Smart: Robust and efficient fine-tuning for pre-trained natural language models through principled regularized optimization. In Annual Conference of the Association for Computational Lingnistics, 2020b.
X. Jiao, Y. Yin, L. Shang, X. Jiang, X. Chen, L. Li, F. Wang. and Q. Liu. Tinybert: Distilling bert for natural language understanding. Preprint arviv: 1909. 10351, 2019.
A. Katharopoulos and F. Fleuret. Not all samples are created equal: Deep learning with importance sampling. In International Conference on Machine Learning, 2018.
A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, 2012.
A. Krizhevsky, V. Nair, and G. Hinton. The cifar-10 dataset. edulkriz/ifaa: himl, 55, 2014.
online: http://www. toronto.
V. Kumar, A. Choudhary, and Cho. Data augmentation using pre-trained transformer models. Preprint 2XXi:22003.02245.5 2020.
J. Li, B. Ziebart, and B. Berger-Wolf. A game-theoretic adversarial approach dynamic network prediction. In Pacific-Asia Conference on Knowledge Discovery and Data Mining, pp. 677-688. Springer, 2018
Lim, Kim, T. Kim, C. Kim, and S. Kim. Fast autoaugment. In Advances in Neural Information Processing Systems, 2019,
T. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár. Focal loss for dense object detection. In IEEE International Conference on Computer Vision, 2017.
Loshchilov and F. Hutter. Decoupled weight decay regularization. In International Conference on Learning Representationss 2018.
A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018.
T. Malisiewicz, A. Gupta, and A. A. Efros. Ensemble of exemplar-svms for object detection and beyond. In International Conference Computer Vision, 2011 1.
J. Martens. New insights and perspectives on the natural gradient method. Preprint 1xXivv1412..199. 2019.
D. Needell, R. Ward, and N. Srebro. Stochastic gradient descent, weighted sampling, and the randomized kaczmarz algorithm. Advances in Neural Information Processing Systems, 2014.
D. Park, Chan, Zhang, Chiu, B. Zoph, D. Cubuk, and Q. V. Le. Specaugment: A simple data augmentation method for automatic speech recognition. In Interspeech, 2019.
A. Radford, J. Wu, R. Child, D. Luan, D. Amodei. and I. Sutskever. Language models are unsupervised multitask learners. OpenAl Blog, 1(8):9, 2019.
A. Raghunathan, M. Xie, F. Yang. J. C Duchi, and P. Liang. Adversarial training can hurt generalization. Preprint arviv 100::1990000033 2019.
10

---

Published as a conference paper at ICLR 2021
M. Ren, W. Zeng, B. Yang, and R. Urtasun. Learning to reweight examples for robust deep learning. In International Conference on Machine Learning, 2018.
J. Shu, Q. Xie, L. Yi, Q. Zhao, S. Zhou, Z. Xu, and D. Meng. Meta-weight-net: Learning an explicit mapping for sample weighting. In Advances in Neural Information Processing Systems, 2019.
Sutskever, O. Vinyals, and Q V. Le. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems, 2014.
T. Tran, T. Pham, G. Carneiro, L. Palmer, and I. Reid. A bayesian data augmentation approach for learning deep models. Advances in Neural Information Processing Systems, 2017.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N Gomez, t. Kaiser, and I. Polosukhin. Attention all you need. In Advances in Neural Information Processing Systems, 2017.
A. Wang, A. Singh, Michael, Hill, O. Levy, and S. R. Bowman. Glue: multi-task benchmark and analysis platform for natural language understanding. In International Conference Learning Representations, 2019.
D. Wang, Yand Yang. That's so annoying!!!: A lexical and frame-semantic embedding based data augmentation approach automatic categorization of annoying behaviors using# petpeeve tweets. In Conference on Empirical Methods in Natural Language Processing, 2015.
J. Wei and K. Zou. Eda: Easy data augmentation techniques for boosting performance on text classification tasks. In Conference on Empirical Methods Natural Language Processing, 2019.
Q. Xie, Z. Dai, F. Hovy, Luong, and Q. V. Le. Unsupervised data augmentation for consistency training. Preprint arXiv:1904. 12848, 2019.
Z. Xie, S. Wang. J. Li, D. Lévy, A. Nie, D. Jurafsky, and A. Y. Ng. Data noising smoothing in neural network language models. In International Conference on Learning Representations, 2017.
Y. Yang, L. Huang, and M. Ma. Breaking the beam search curse: A study of (re-) scoring methods and stopping criteria for neural machine translation. In Conference Empirical Methods in Natural Language Processing, 2018.
H. Zhang, M. Cisse. Y. N. Dauphin, and D. Lopez-Paz. mixup: Beyond empirical risk minimization.
In International Conference on Learning Representations, 2018.
X. Zhang, J. Zhao, and Y. LeCun. Character-level convolutional networks for text classification. In Advances in Neural Information Processing Systems, 2015.
P. Zhao and T. Zhang. Accelerating minibatch stochastic gradient descent using stratified sampling. preprint arXiv: 1405.3000 2014.
C. Zhu, Y. Cheng, Z. Gan, S. Sun, T. Goldstein, and J. Liu. Freelb: Enhanced adversarial training for natural language understanding. International Conference on Learning Representations, 2020.

---

Published as a conference paper at ICLR 2021
A PROOF OF THEOREM
Proof. For any given æ; and (æx;), aim to find Pa(: æ;)) on (xx;) such that
Σ
Po(z (:){([(z)).
ApPo(z log (IB(2)))Po(2 (x)) mx))
P s(e()
(10)
s.t.
Σ
Po(z xx)) 1.
BEB(2.)
Since the objective is convex, by Lagrange multiplier method, let
Σ =EB(a()
Po(z m:))({o(z))) ApPP(z a:)) log ( (BB())) Po(z ax))
(11)
Σ
Po(z xi) -1
=EB(a,)
From Vr,C((P.NN) VxC(IPo, A) O, for any pairs of zu. 2o B(x;), have ((fo(zu); Vzu) ) (rog (mx))) + log Po x.)) = ((oo(z))) Vz.) Ar(log B(x))] + 1 + log (z. x;)))
(12)
Hence we have
[((o(Ze). {(Folzu); Uau
Po x.)) Po(su x.) exp
exp
(13)
Ap
Summing over 2, B(x;), have
exp [(fo(zo), Dz.) e(folzu), )zu))
Po(z. xx)
Σ
(14)
dp
The proof completes.
B MMEL ON LARGE-SCALE DATASET
In this section, we evaluate the proposed method MMEL on large-scale image classification task ImageNet (Deng et al. 2009).
Data. ImageNet is benchmark dataset which contains colorful images with over 1 million training samples and 50000 validation samples from 1000 categories.
Setup. The model we used is ResNet for ImageNet with three different depths (He et al. 2016). All these experiments are conducted 100 epochs, and the learning rate decays at epochs 30, 60, and 90. We set batch size 256, and [B(x,)] 10 for each æ;, The other experimental settings follow Section 4.] expect for the following hyperparameters. We compare the proposed method with "Baseline(DAA).
The results are shown in Table From the results, the proposed MMEL-H improves the performance of the model for all three depths. However, the proposed MMEL-S is beaten by the baseline method. We speculate this due to the relatively larger proportion of inaccurate prediction of original training samples on the large-scale dataset. More specifically, as in equation (8). for each augmented sample z E B(x;), the proposed MMEL-S encourages the model to the output of original training sample fo(x,) However, the accuracy of the original training samples in the ImageNet dataset can not reach 100% e.g., about 80% for ResNet50 on ImageNet. The inaccurate prediction fo(x;) can be misleading supervision for augmented sample B(x,). leading to degraded performance of the proposed MMEL-S. Thus, we suggest using the MMEL-H the accuracy of the original training samples is relative low.
Main Results.
12

---

Published as a conference paper at ICLR 2021
Table 5: Performance of ResNet on ImageNet.
dataset
Model
Baseline(DA)
MMEL-H
MMEL-S
ResNet18
69.76
68.52(-1.24) 70.48(+0.72)
TmageNet
ResNet34
73.30
72.33(40.97) 74.38(+1.08)
ResNe150
76.15
76.53(+0.38)
74.82(-1.33)
C GENERATING AUGMENTED SAMPLES FOR TEXTUAL SAMPLES
In this section, we elaborate the procedure of generating augmented sentences using greedy-based and beam- method for sequence. For each original input sentence, we randomly mask k tokens (which is obtained by rounding the product masking ratio and length of the sequence nearest number) and then we do a forward propagation of the BERT to predict the tokens in those masked positions using greedy search. The detailed procedure is shown in Algorithm 2 We also use beam search Yang et al. 2018) generate augmented data. The details of beam search can be referred to Yang et al., 2018). y. For sentence- pair tasks, we treat the two sentences separately and generate augmented samples for each of them.
Algorithm 2 Augmented Sample Generation by Greedy Search
Input: Prantee language model Be BertMell, original sentence æ;, number of augmented samples [B(x) 1, number of masked tokens k.
Output: Augmented samples (xx) {z1, Ø2, ,²
1: Randomly sample & positions {PI, } and get mmakk-
2: for = 1.,,,...
do
D Generate the i-th augmented sample
3:
f- Tmask.
4: 2:(p1] the ith most likely word predicted by BertModel
5: for in {2, 3. ... k] do
6: z; [p,] +- the most likely word predicted by PertModel
7: end for
8:
end for
In the following, we vary the factors that may affect the quality of the generated augmented samples. These factors include
1. The number of masked tokens, which equals the replacement proportion multiplied with the sentence length. This affects the diversity of augmented samples, i.e., replacing large proportion of tokens makes the augmented sample less similar to the original one.
2. Treating the two sentences separately in sentence-pai tasks when generating augmented examples, or concatenate them single sentence;
Different generation methods like greedy search (Algorithm) and beam search.
The results are shown in Table As can be seen, compared with Baseline without data augmentation, MMEL-H and MMEL-S under all hyperparameter configurations have higher accuracy, showing the efficacy of data augmentation and the proposed reweighting strategy. There is no significant difference in using greedy search or beam search to generate the augmented samples. In this natural understanding task, training with augmented samples generated with proper larger replacement proportion (i.e., larger diversity) has slightly better performance. For sentence-pain tasks, treating the two sentences separately and generate augmented samples for each of them has slightly better performance. In the experiments in Section 4.2) we use Greedy search, masking proportion 0.4, and generate augmented sentence for each sentence in sentence-pair tasks.
D HYPERPARAMETERS FOR THE EXPERIMENT ON THE GLUE BENCHMARK.
The optimizer we used is AdamW (Loshchilov & Hutter 2018 The hyperparameters of BERT BASE model are listed in Table
13

---

Published as a conference paper at ICLR 2021
Table Ablation study on generating augmented samples on the GLUE benchmark. Development set results are reported.
Method
Separate sentence-pair
Replacement prepartion
CoLA
SET
MRPC
STS-B
QQ2
MNLL m//mm
QNLI
RTE
Avg
Bascline
59.0
87.5
89.8
84.6/8500
71.1
83.7
Greedy
True
0.2
60.2
93.1
87.7
90.0
91.5
85.4785.6
92.4
71.5
84.1
Gready
62.8
87.5
85,4/85.6
84.5
Greedy
False
0.2
60.2
93.6
87.0
89.8
91.5
85 85.4/85.7
92.5
69.3
84.0
MMEL-H
Greedy
False
0.4
61.8
92.7
88.0
90.0
91.5
85.388.55
92.2
72.2
84.4
Beam
True
0.2
60.9
93.2
87.0
90.0
91.4
85,5/85.5
92.4
71.1
84.0
Beam
True
0.4
60.8
93.1
88.0
90.3
91.3
85,3/85,5
92.3
70.3
84.5
0,2
60.0
86.7
89.8
91.5
92.3
Beam
False
0.4
60.8
93.1 1
88.0
90.0
91.4
88.38
92.3
70.4
84.1
Groody
61.0
93.
89.5
91.6
85.3/83.
84.2
Greedy
Trae
0..4
61.7
9..8
89.2
90.2
91.6
85.0/85.5
92.5
73.3
84.7
Greedy
MMEL-S
Greedy
False
0.4
61.8
93.0
87.7
90.0
91.6
85. 85.1/85.6
92.5
72.2
84.4
Beam
0.2
62.0
85 95,4/85.9
92.5
84.2
Beam
True
0.4
61.0
93.0
87.7
90.2
91.4
85.2/85.5
92.2
73.6
84.0
Beam
False
0.2
62.0
86.7
89.7
91.6
85.388599
92.4
70.0
84.1
Beam
False
0.4
61.0
93.1
88.2
89.7
91.4
85,2/85.7
92.2
72.2
84.3
Table Hyperparameters of the BERT BASE model.
Hyperparam
MMEL-H
MMEL-S
Learning Rate
3e-5
3e-5
Batch Size
32
Weight Decay
0
0
Hidden Layer Dropout Rate
(0, 0.1)
[0, 0.1)
Attention Probability Dropout Rate
(0. 0.1)
[0, 0.1)
Max Epochs (MNLI, QQP)
3
Max Epochs (Others)
10
10
Learning Rate Decay
Linearly
Linearly
Warmup Ratio
0
X
1.0
1.0
Ap
1.0
1.0
Number of Candidates (B((x)))
5
14


Reviews:
[{"review_id": "sfDtyrPHl6_", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\nThis paper proposes a simple scheme for training with multiple augmentations of training data in one iteration and reweighting the instances by their relative loss. As authors note in their related works, the idea of reweighting examples based on their relative loss has been widely studied in a variety of machine learning problems. In contrast, this work proposes the reweighting only within augmentations of a single sample. They derive their particular reweighting scheme by proposing an alternative risk (Eq. 3). The new objective is a function of both model parameters and the distribution of augmentations. They propose to find the model parameters that minimize the alternative risk for the hardest distribution of augmentations that maximizes their alternative risk (Eq. 4). Then they consider the distribution of augmentations that are a function of model parameters and input and show that for fixed model parameters, the optimal distribution on a fixed finite set of augmentations is determined by the softmax on the loss of the model for each augmented input. In section 3.2, they propose two variations of their loss using the ground-truth label to evaluate the loss (hard loss) versus using the prediction of the model for the original raw input (soft loss). In section 3.3, they propose specific considerations for augmenting text data. They provide experiments on image and text data with ablations studies.\n\nPros:\n- Proposed methods are particularly good on large models (resnet44, resnet56)\n- Figure 2 is particularly interesting because the curves for proposed methods seem less noisy in multiple datasets (CoLA, SST-2 mrpc (only soft), qnli).\n- The method might show more advantage if tested in low data settings or small mini-batches. Have authors tried any setting with small training data? Or smaller mini-batches?\n\nCons:\n- The method does not always beat DA+UNI that uses multiple augmentations but weights them uniformly. Although one could argue DA+UNI is also an interesting contribution as it can be more efficient than DA+Long for small mini-batches.\n- Tables: no standard deviation is given? In particular, in Table 1, 0.5 can easily be within the standard deviation.\n- Table 1: DA+UNI should have the same time as MMEL but for resnet56 it takes 7h while MMEL takes 4.5h to train\n- It is not clear how the hyperparameter lambda is tuned? Was it done using cross-validation? In the appendix the lambda is given as 1. Is that the best value?\n- Table 3: Results on CoLA are good but others not significant? Again, not clear based on missing standard deviation.\n\nAdditional Notes:\n- Can you clarify the following sentence: “Remark 3: If we ignore the KL divergence term in equation (3), due to the equivalence of minimizing cross-entropy loss and MLE loss...”\n- Is Eq 7. just Eq 5 but for a single example?\n- Figure 1 is not really descriptive enough. Maybe make it one figure and use color and line style to show the difference.\n- DA+UNI: does the implementation use bigger batch size to perform computations in parallel?\n\nTypos:\nSection 3.1: an uniform -> a uniform\nRemark 2: regularizedr -> regularizer\n\n==============\nAfter rebutall:\nI thank the authors and appreciate addressing all my concerns. I'm glad that the additional experiments with smaller mini-batches and smaller training set size provide more supporting evidence for the method. I encourage the authors to point to these results in the main body.\n\nOne more minor suggestion is regarding the sentence in Remark 3. The way I read the sentence the \"equivalence of minimizing CE loss and MLE loss\" is the reason we can remove the KL divergence term. But the authors' response seems to say it is for the 3rd part of the sentence. The authors' might want to rewrite that sentence to make it clear.", "extra_scores": {}, "extra_text": {}}, {"review_id": "nsNEh5C0Lj-", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "##########################################################################\n\nSummary:\n \nThe paper proposes a novel data augmentation method. In particular, it proposes a reweighted loss function that allows to find the optimal weighting of the augmented samples. The approach is tested on standard image and \nlanguage tasks and compared to mulitple alternative approaches.\n\n##########################################################################\n\nReasons for score: \n \n\nOverall, I vote for accepting. The approach is interesting, well motivated and clearly formalized. However the experiments raise some questions which I would like the authors to comment on in the rebuttal. \n \n\n##########################################################################\n\nPros: \n \n1. The paper has an interesting take on the important topic of data augmentation  \n\n2. It proposes a clever and theoretically well motivated way to incorporate augmented samples. \n\n3. The base line models are chosen reasonable. \n\n4. The paper is clearly structured and well written.\n\n##########################################################################\n\nCons: \n \n1. One major concern is the performance comparison of varying numbers of augmented samples. Table 2 shows that often a smaller amount of augmented samples perform better. However, if the reweighting is indeed optimal, adding augmented samples should not hurt the performance.  \n\n\n2. It is not clear when and why to use the hard or soft loss. In addition with the varying performance dependent on the number of augmentation samples this adds additional hyperparameters. The increase in performance might not be worth the increase in training time (see point 3).\n\n\n3. Are the results significant? \n \n\n##########################################################################\n\nQuestions during rebuttal period: \n \n\nPlease address and clarify the cons above \n \n\n#########################################################################\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "OD0GqcPoeOR", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "##########################################################################\n\nSummary:\n\n \nThe paper investigate an interesting problem, improving generalization performance by leveraging augmented data.\n\n##########################################################################\n\nReasons for score: \n\n \nOverall, my vote lies on borderline and my main concern is the novelty of the work.\n\n \n##########################################################################\nPros: \n1. The paper studies an important problem and propose a method that is a mixture of weighted augmentation and adversarial loss.\n2. The paper is written well and easy to follow.\n- Figure 1 is a nice representation to help the reader to understand the loss function better.\n3. The paper investigates the performance of the proposed method on two domains: computer vision and natural language processing. The authors also provide ablation studies to evaluate the role of parameters.\n \n##########################################################################\n\nCons: \n\n \n1- I have two main concerns:\na) The novelty of the method is questionable for me as I have already seen a very close idea in the following papers:\n[1] https://ieeexplore.ieee.org/abstract/document/8658998\n[2] https://arxiv.org/pdf/1712.04621.pdf\n[3]https://arxiv.org/abs/1907.12934\nIn [1], the min-max (adversarial) framework used and the distribution over the data provides weighting augmentation.\nIn [2], the effectiveness of data augmentation in object recognition has been discussed.\nIn [3], the min max entropy has been leveraged for weakly supervised (pixel-level) localization.\n....\nIt is unfortunate that this papers were not even cited.\nb) Generalization aspect.\nThe authors of the paper presents their main objective to provide generalization while they have not provided comprehensive experiments or discussion to address this concern.\n\n2- Although the proposed method provides experiments on computer vision and natural language processing, I still suggest the authors to conduct more experiments considering more datasets from computer vision domain.\n\n3-It would be great if the author can what would be the advantages/ weaknesses of their approach with the references. As the proposed approach has high similarity with the previous works, the minimum requirement is reporting more experiments and compare their method with exist methods. This extra study would present how their approach affect the performance.\n \n##########################################################################\n\nQuestions during rebuttal period: \n\n \nPlease address and clarify the cons above \n\n \n#########################################################################\nAfter rebuttal:\n\nDear Authors, Thanks for providing more details. I believe more discussion and experiments are required to present the difference of you method. As you have mentioned, one difference is in considering summation rather than maximization, so it would be required to know what would be the advantages/ weaknesses of this difference. How does it make any impact on the performance? I would increase my score considering the closed-form solution as a nice contribution and requiring more experiments and analysis on the discussed references.\n\nThanks!\n\n\n ", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Dear authors,

The reviewers appreciated the insights provided by your paper and the strong results. Congratulations.
I encourage you to address the other points raised to make your final submission as complete as possible.
Decision: Accept (Poster)