R EWEIGHTING A UGMENTED S AMPLES BY M INIMIZING THE M AXIMAL E XPECTED L OSS
Anonymous authors
Paper under double-blind review
A BSTRACT
Data augmentation is an effective technique to improve the generalization of deep neural networks.
However, previous data augmentation methods usually treat the augmented samples equally without considering their individual impacts on the model.
To address this, for the augmented samples from the same training example, we propose to assign different weights to them.
We construct the maximal expected loss which is the supremum over any reweighted loss on augmented samples.
Inspired by adversarial training, we minimize this maximal expected loss (MMEL) and obtain a simple and interpretable closed-form solution: more attention should be paid to augmented samples with large loss values (i.e., harder examples).
Minimizing this maximal expected loss enables the model to perform well under any reweighting strategy.
The proposed method can generally be applied on top of any data augmentation methods.
Experiments are conducted on both natural language understanding tasks with token-level data augmentation, and image classiﬁcation tasks with commonly-used image augmentation techniques like random crop and horizontal ﬂip.
Empirical results show that the proposed method improves the generalization performance of the model.
I NTRODUCTION
Deep neural networks have achieved state-of-the-art results in various tasks in natural language processing (NLP) tasks (Sutskever et al., 2014; Vaswani et al., 2017;
Devlin et al., 2019) and computer vision (CV) tasks (He et al., 2016; Goodfellow et al., 2016).
One approach to improve the generalization performance of deep neural networks is data augmentation (Xie et al., 2019; Jiao et al., 2019; Cheng et al., 2019; 2020).
However, there are some problems if we directly incorporate these augmented samples into the training set.
Minimizing the average loss on all these samples means treating them equally, without considering their different implicit impacts on the loss.
To address this, we propose to minimize a reweighted loss on these augmented samples to make the model utilize them in a cleverer way.
Example reweighting has previously been explored extensively in curriculum learning (Bengio et al., 2009; Jiang et al., 2014), boosting algorithms (Freund &
Schapire, 1999), focal loss (Lin et al., 2017) and importance sampling (Csiba & Richtárik, 2018).
However, none of them focus on the reweighting of augmented samples instead of the original training samples.
A recent work (Jiang et al., 2020a) also assigns different weights on augmented samples.
But weights in their model are predicted by a mentor network while we obtain the weights from the closed-form solution by minimizing the maximal expected loss (MMEL).
In addition, they focus on image samples with noisy labels, while our method can generally be applied to also textual data as well as image data.
Tran et al.
(2017) propose to minimize the loss on the augmented samples under the framework of Expectation-Maximization algorithm.
But they mainly focus on the generation of augmented samples.
Unfortunately, in practise there is no way to directly access the optimal reweighting strategy.
Thus, inspired by adversarial training (Madry et al., 2018), we propose to minimize the maximal expected loss (MMEL) on augmented samples from the same training example.
Since the maximal expected loss is the supremum over any possible reweighting strategy on augmented samples’ losses, minimizing this supremum makes the model perform well under any reweighting strategy.
More importantly, we derive a closed-form solution of the weights, where augmented samples with larger training losses have larger weights.
Intuitively, MMEL allows the model to keep focusing on augmented samples that are harder to train.
The procedure of our method is summarized as follows.
We ﬁrst generate the augmented samples with commonly-used data augmentation technique, e.g., lexical substitution for textual input (Jiao et al., 2019), random crop and horizontal ﬂip for image data (Krizhevsky et al., 2012).
Then we explicitly derive the closed-form solution of the weights on each of the augmented samples.
After that, we update the model parameters with respect to the reweighted loss.
The proposed method can generally be applied above any data augmentation methods in various domains like natural language processing and computer vision.
Empirical results on both natural language understanding tasks and image classiﬁcation tasks show that the proposed reweighting strategy consistently outperforms the counterpart of without using it, as well as other reweighting strategies like uniform reweighting.
R ELATED W ORK
Data augmentation.
Data augmentation is proven to be an effective technique to improve the generalization ability of various tasks, e.g., natural language processing (Xie et al., 2019; Zhu et al., 2020; Jiao et al., 2019), computer vision (Krizhevsky et al., 2014), and speech recognition (Park et al., 2019).
For image data, baseline augmentation methods like random crop, ﬂip, scaling, and color augmentation (Krizhevsky et al., 2012) have been widely used.
Other heuristic data augmentation techniques like Cutout (DeVries & Taylor, 2017) which masks image patches and Mixup (Zhang et al., 2018) which combines pairs of examples and their labels, are later proposed.
Automatically searching for augmentation policies (Cubuk et al., 2018; Lim et al., 2019) have recently proposed to improve the performance further.
For textual data, Zhang et al. (2015); Wei & Zou (2019) and
Wang (2015) respectively use lexical substitution based on the embedding space.
Jiao et al.
Cheng et al.
; Kumar et al.
(2020) generate augmented samples with a pre-trained language model.
Some other techniques like back translation (Xie et al., 2019), random noise injection (Xie et al., 2017) and data mixup (Guo et al., 2019; Cheng et al., 2020) are also proven to be useful.
Adversarial training.
Adversarial learning is used to enhance the robustness of model (Madry et al., 2018), which dynamically constructs the augmented adversarial samples by projected gradient descent across training.
Although adversarial training hurts the generalization of model on the task of image classiﬁcation (Raghunathan et al., 2019), it is shown that adversarial training can be used as data augmentation to help generalization in neural machine translation (Cheng et al., 2019; 2020) and natural language understanding (Zhu et al., 2020; Jiang et al., 2020b).
Our proposed method differs from adversarial training in that we adversarially decide the weight on each augmented sample, while traditional adversarial training adversarially generates augmented input samples.
In (Behpour et al., 2019), adversarial learning is used as data augmentation in object detection.
The adversarial samples (i.e., bounding boxes that are maximally different from the ground truth) are reweighted to form the underlying annotation distribution.
A similar method is also proposed in dynamic network prediction (Li et al., 2018).
However, besides the difference in the model and task, their training objective and the resultant solution are also different from ours.
Sample reweighting.
Minimizing a reweighted loss on training samples has been widely explored in literature.
Curriculum learning (Bengio et al., 2009; Jiang et al., 2014) feeds ﬁrst easier and then harder data into the model to accelerate training.
Zhao & Zhang (2014); Needell et al.
Csiba & Richtárik (2018); Katharopoulos & Fleuret (2018) use importance sampling to reduce the variance of stochastic gradients to achieve faster convergence rate.
Boosting algorithms (Freund & Schapire, 1999) choose harder examples to train subsequent classiﬁers.
Similarly, hard example mining (Malisiewicz et al., 2011) downsamples the majority class and exploits the most difﬁcult examples.
Focal loss (Lin et al., 2017; Goyal & He, 2018) focuses on harder examples by reshaping the standard cross-entropy loss in object detection.
Ren et al.
; Jiang et al.
Shu et al.
(2019) use meta-learning method to reweight examples to handle the noisy label problem.
Unlike all these existing methods, in this work, we reweight the augmented samples’ losses instead of training samples.
MINIMIZE THE MAXIMAL E XPECTED L OSS
In this section, we derive our reweighting strategy on augmented samples from the perspective of maximal expected loss.
We ﬁrst give a derivation of the closed-form solution of the weights on augmented samples.
Then we describe two kinds of loss under this formulation.
Finally, we give the implementation details using the natural language understanding task as an example.
3. W HY M AXIMAL E XPECTED L OSS
Consider a classiﬁcation task with N training samples.
For the i -th training sample x i , its label is denoted as y x i .
Let f θ ( · ) be the model with parameter θ which outputs the classiﬁcation probabilities.
(cid:96) ( · , · ) denotes the loss function, e.g. the cross-entropy loss between outputs f θ ( x i ) and the groundtruth label y x i .
Given an original training sample x i , the set of augmented samples generated by some method is B ( x i ) .
Without loss of generality, we assume x i ∈ B ( x i ) .
The conventional training objective is to minimize the loss on every augmented sample z in B ( x i ) as
[Equation]
where y z is the label of z ∈ B ( x i ) , and can be different with y x i .
| B ( x i ) | is the number of augmented samples in B ( x i ) , which is assumed to be ﬁnite.
in equation (1), for each x i , the weights on each of its augmented samples are the same (i.e., 1 / | B ( x i ) | ).
However, different samples have different implicit impacts on the loss, and we can assign different weights on them to facilitate training.
Note that computing the weighted sum of losses of each augmented sample in B ( x i ) can be viewed as taking expectation of loss on augmented samples z ∈ B ( x i ) under a certain distribution.
When the augmented samples generated from the same training sample are drawn from a uniform distribution, the loss in equation (1) can be rewritten as
[Equation]
where the Kullback–Leibler (KL) divergence KL ( P U ( · | x i ) (cid:107) P U ( · | x i )) equals zero.
Here P
U ( · | x i ) denotes the uniform distribution on B ( x i ) .
When the augmented samples are drawn from a more general distribution P B ( · | · ) 1 instead of the uniform distribution, we can generalize P U ( · | · ) here to some other conditional distribution P B .
[Equation]
Remark 1.
When P B ( · | x i ) reduces to the uniform distribution P U ( · | x i ) for any x i , since
KL ( P U ( · | x i ) (cid:107) P U ( · | x i ))
= 0 , the objective in equation (3) reduces to the one in equation (1) .
The KL divergence term in equation (3) is used as a regularizer to encourage P B close to P U (see
Remark 2).
From equation (3), the conditional distribution P B determines the weights of each augmented sample in B ( x i ) .
There may exist an optimal formulation of P B in some regime, e.g. corresponding to the optimal generalization ability of model.
Unfortunately, we can not explicitly characterize such an unknown optimal P B .
To address this, we borrow the idea from adversarial training (Madry et al., 2018) and minimize the maximal reweighted loss on augmented samples.
Then, the model is guaranteed to perform well under any reweighting strategy, including the underlying optimal one.
Speciﬁcally, let the conditional distribution P B be P ∗ θ = arg sup P B R
θ ( P B ) .
Our objective is to minimize the following reweighted loss
[Equation]
The following Remark 2 discusses about the KL divergence term in equation (3).
Remark 2.
Since we take a supremum over P B in equation (4) , the regularizer KL ( P B (cid:107) P U ) encourages P B to be close to P U because it reaches the minimal value zero when P B = P U .
Thus the regularizer controls the diversity among the augmented samples by constraining the discrepancy between P B and uniform distribution P U , e.g., a larger λ
P promotes a larger diversity among the augmented samples.
The following Theorem 1 gives the explicit formulation of R θ ( P ∗ θ ) .
Theorem 1.
Let R θ ( P B ) and R θ
( P ∗ θ ) be deﬁned in equation (1) and (4) , then we have
[Equation]
where
[Equation]
where Softmax z ( 1 λ P (cid:96) ( f θ ( B ( x i )) , y B ( x i ) )) represents the output probability of z for vector ( 1 λ P (cid:96) ( f θ ( z 1 ) , y z 1 ) , · · · , 1 λ P (cid:96) ( f θ ( z | B ( x i ) | ) , y | B ( x i ) | )) .
Remark 3.
If we ignore the KL divergence term in equation (3) , due to the equivalence of minimizing cross-entropy loss and MLE loss (Martens, 2019), the proposed MMEL also falls into the generalized
Expectation-Maximization (GEM) framework (Dempster et al., 1977).
Speciﬁcally, given a training example, the augmented samples of it can be viewed as latent variable, and any reweighting on these augmented samples corresponds to a speciﬁc conditional distribution of these augmented samples given the training sample.
In the expectation step (E-step), we explicitly derive the closed-form solution of the weights on each of these augmented samples according to (6) .
In the maximization step, since there is no analytical solution for deep neural networks, following (Tran et al., 2017), we update the model parameters with respect to the reweighted loss by one step of gradient descent.
The proof of this theorem can be found in Appendix A. From Theorem 1, the loss of it decides the weight on each augmented sample z ∈ B x i , and the weight is normalized by Softmax over all augmented samples in B x i .
The reweighting strategy allows more attention paid to augmented samples with higher loss values.
The strategy is similar to those in (Lin et al., 2017; Zhao & Zhang, 2014) but they apply it on training samples.
3. T WO T YPES OF L OSS
For augmented sample z ∈ B ( x i ) , instead of computing the discrepancy between the output probability f θ ( z ) and the hard label y z as in equation (5), one can also compute the discrepancy between f θ ( z ) and the “soft” probability f θ ( x i ) in the absence of ground-truth label on augmented samples as in (Xie et al., 2019).
In the following, We use superscript “hard" for the loss in equation (5) as
[Equation]
to distinguish with the following objective which uses the “soft probability”:
[Equation]
The two terms in R soft θ ( P ∗ θ , x i ) respectively correspond to the loss on original training samples x i and the reweighted loss on the augmented samples.
The reweighted loss promotes a small discrepancy between the augmented samples and the original training sample.
λ T > 0 is the coefﬁcient used to balance the two loss terms, and P ∗ θ ( z | x i ) is deﬁned similar to (6) as
[Equation]
The two losses are shown in Figure 1.
Summing over all the training samples, we get the two kinds of reweighted training objectives.
𝒙𝒙 𝒊𝒊 𝒛𝒛𝒛𝒛𝒛𝒛 𝐵𝐵 − 𝑓𝑓 𝜃𝜃 ( 𝒛𝒛 2 ) ℓ ( 𝑓𝑓 𝜃𝜃 ( 𝒛𝒛 1 ), 𝑓𝑓 𝜃𝜃 ( 𝒙𝒙 𝑖𝑖 )) 𝑓𝑓 𝜃𝜃 ( 𝒛𝒛 1 ) 𝑓𝑓 𝜃𝜃 ( 𝒛𝒛 𝐵𝐵 −1 ) ℓ ( 𝑓𝑓 𝜃𝜃 ( 𝒛𝒛 2 ), 𝑓𝑓 𝜃𝜃 ( 𝒙𝒙 𝑖𝑖 ))
ℓ ( 𝑓𝑓 𝜃𝜃 ( 𝒛𝒛 | 𝐵𝐵 | −1 ), 𝑓𝑓 𝜃𝜃 ( 𝒙𝒙 𝑖𝑖 )) …
× 𝑝𝑝𝑅𝑅 𝜃𝜃𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠 ( 𝑃𝑃 𝜃𝜃∗ , 𝒙𝒙 𝒊𝒊 ) ×
𝑝𝑝× 𝑝𝑝 𝐵𝐵 − 𝑓𝑓 𝜃𝜃
( 𝒙𝒙 𝑖𝑖 ) × ℓ ( 𝑓𝑓 𝜃𝜃 ( 𝒙𝒙 𝑖𝑖 , 𝑦𝑦 𝒙𝒙 𝑖𝑖 ) 𝒙𝒙 𝒊𝒊 𝒛𝒛𝒛𝒛𝒛𝒛 𝐵𝐵 − 𝑓𝑓 𝜃𝜃 ( 𝒛𝒛 2 ) ℓ ( 𝑓𝑓 𝜃𝜃 ( 𝒛𝒛 1 ), 𝑦𝑦 𝒛𝒛 1 ) 𝑓𝑓 𝜃𝜃 ( 𝒛𝒛 1 ) 𝑓𝑓 𝜃𝜃 ( 𝒛𝒛 𝐵𝐵 −1 ) ℓ ( 𝑓𝑓 𝜃𝜃 ( 𝒛𝒛 2 ), 𝑦𝑦 𝒛𝒛 2 ) ℓ ( 𝑓𝑓 𝜃𝜃 ( 𝒛𝒛 𝐵𝐵 −1 ), 𝑦𝑦 𝒛𝒛 3 ) … × 𝑝𝑝𝑅𝑅 𝜃𝜃ℎ𝑎𝑎𝑎𝑎𝑎𝑎 ( 𝑃𝑃 𝜃𝜃∗ , 𝒙𝒙 𝒊𝒊 ) × 𝑝𝑝× 𝑝𝑝 𝐵𝐵 − 𝑓𝑓 𝜃𝜃 ( 𝒙𝒙 𝑖𝑖 ) × 𝑝𝑝 𝐵𝐵 ℓ ( 𝑓𝑓 𝜃𝜃 ( 𝒙𝒙 𝑖𝑖 , 𝑦𝑦 𝒙𝒙 𝑖𝑖 )
(a) MMEL-H. 𝒙𝒙 𝒊𝒊 𝒛𝒛𝒛𝒛𝒛𝒛 𝐵𝐵 − 𝑓𝑓 𝜃𝜃 ( 𝒛𝒛 2 ) ℓ ( 𝑓𝑓 𝜃𝜃 ( 𝒛𝒛 1 ), 𝑓𝑓 𝜃𝜃 ( 𝒙𝒙 𝑖𝑖 ))
𝑓𝑓 𝜃𝜃 ( 𝒛𝒛 1 ) 𝑓𝑓 𝜃𝜃 ( 𝒛𝒛 𝐵𝐵 −1 ) ℓ ( 𝑓𝑓 𝜃𝜃 ( 𝒛𝒛 2 ), 𝑓𝑓 𝜃𝜃 ( 𝒙𝒙 𝑖𝑖 ))
ℓ ( 𝑓𝑓 𝜃𝜃 ( 𝒛𝒛 | 𝐵𝐵 | −1 ), 𝑓𝑓 𝜃𝜃 ( 𝒙𝒙 𝑖𝑖 )) …
× 𝑝𝑝𝑅𝑅 𝜃𝜃𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠 ( 𝑃𝑃 𝜃𝜃∗ , 𝒙𝒙 𝒊𝒊 ) ×
𝑝𝑝× 𝑝𝑝 𝐵𝐵 − 𝑓𝑓 𝜃𝜃
( 𝒙𝒙 𝑖𝑖 ) × ℓ ( 𝑓𝑓 𝜃𝜃 ( 𝒙𝒙 𝑖𝑖 , 𝑦𝑦 𝒙𝒙 𝑖𝑖 ) 𝒙𝒙 𝒊𝒊 𝒛𝒛𝒛𝒛𝒛𝒛 𝐵𝐵 − 𝑓𝑓 𝜃𝜃 ( 𝒛𝒛 2 ) ℓ ( 𝑓𝑓 𝜃𝜃 ( 𝒛𝒛 1 ), 𝑦𝑦 𝒛𝒛 1 ) 𝑓𝑓 𝜃𝜃 ( 𝒛𝒛 1 ) 𝑓𝑓 𝜃𝜃 ( 𝒛𝒛 𝐵𝐵 −1 ) ℓ ( 𝑓𝑓 𝜃𝜃 ( 𝒛𝒛 2 ), 𝑦𝑦 𝒛𝒛 2 ) ℓ ( 𝑓𝑓 𝜃𝜃 ( 𝒛𝒛 𝐵𝐵 −1 ), 𝑦𝑦 𝒛𝒛 3 ) … × 𝑝𝑝𝑅𝑅 𝜃𝜃ℎ𝑎𝑎𝑎𝑎𝑎𝑎 ( 𝑃𝑃 𝜃𝜃∗ , 𝒙𝒙 𝒊𝒊 ) × 𝑝𝑝× 𝑝𝑝 𝐵𝐵 − 𝑓𝑓 𝜃𝜃 ( 𝒙𝒙 𝑖𝑖 ) × 𝑝𝑝 𝐵𝐵 ℓ ( 𝑓𝑓 𝜃𝜃 ( 𝒙𝒙 𝑖𝑖 , 𝑦𝑦 𝒙𝒙 𝑖𝑖 ) (
[Figure]
) MMEL-S.
Algorithm
Minimize the Maximal Expected Loss
MMEL
Input: Training set { ( x 1 , y x 1 ) , · · · , ( x N , y x N ) } , batch size S , learning rate η , number of training iterations T , R θ equals R hard θ or R soft θ .
1: for i in { 1 , 2 , · · · , N } do (cid:46) generate augmented samples 2:
Generating B ( x i ) using some data augmentation method.
3: end for 4: for t = 1 , · · · , T do (cid:46) minimize the maximal expected
loss
Randomly sample a mini-batch from training set.
Fetch the augmented samples B
B
Compute P according or
Update model parameters
9: end for
3. E XAMPLE : MMEL I MPLEMENTATION ON N ATURAL L ANGUAGE U NDERSTANDING T ASKS
In this section, we elaborate on implementing the proposed method using textual data in natural language understanding tasks as an example.
Our method is separated into two phases.
In the ﬁrst phase, we generate augmented samples.
Then in the second phase, with these augmented samples, we update the model parameters under these augmented samples with respect to the hard reweighted loss (7) or the soft counterpart (8).
The generation and training procedure can be decoupled, and the augmented samples are ofﬂine generated in the ﬁrst phase by only once.
On the other hand, in the second phase, since we have the explicit solution of weights on augmented samples and the multiple forward and backward passes on these augmented samples can be computed in parallel, the whole training time is similar to the regular training counterpart for an appropriate number of augmented samples.
The whole training process is shown in Algorithm 1.
Generation of Textual Augmented Data.
Various methods have been proposed to generate augmented samples for textual data.
Recently, large-scale pre-trained language models like BERT (Devlin et al., 2019) and GPT-2 (Radford et al., 2019) learn contextualized representations and have been used widely in generating high-quality augmented sentences (Jiao et al., 2019; Kumar et al., 2020).
In this paper, we use a pre-trained BERT trained from masked language modeling to generate augmented samples.
For each original input sentence, we randomly mask k tokens.
Then we do a forward propagation of the BERT to predict the tokens in those masked positions by greedy search.
Details can be found in Algorithm 2 in Appendix B.
Mismatching Label.
For R hard θ in equation (7), the loss term (cid:96) ( f θ ( z ) , y z ) on augmented sample z ∈ B ( x i ) for some x i relies on its label y z .
Unlike image data, where conventional augmentation methods like random crop and horizontal ﬂip of an image do not change its label, substituting even one word in a sentence can drastically change its meaning.
For instance, suppose the original sentence is “She is my daughter” , and the word “She” is masked.
The top 5 words predicted by the pre-trained
BERT are “This, She, That, It, He”.
Apparently, for the task of linguistic acceptability task, replacing “She” with “He” can change the label from linguistically “acceptable” to “non-acceptable”.
Thus for textual input, for the term (cid:96) ( f θ ( z ) , y z ) in hard loss (7), instead of directly setting y z as y x
i
(Zhu et al., 2020), we replace y z with the output probability of a trained teacher model.
On the other hand, for the soft loss in equation (8), if an augmented sample z ∈ B ( x i ) is predicted to a different class from x i by the teacher model, it is unreasonable to still minimize the discrepancy between f θ ( z ) and f θ ( x i ) .
In this case, we replace f θ ( x i ) in the loss term λ T (cid:80) z ∈ B ( x i );
z (cid:54) =
x i P ∗ θ ( z | x i ) (cid:96) ( f θ ( z ) , f θ ( x i )) with the output probability from the teacher model.
E XPERIMENTS
In this section, we evaluate the efﬁcacy of the proposed MMEL algorithm with both hard loss (MMEL-H) and soft loss (MMEL-S).
Experiments are conducted on both the image classiﬁcation tasks CIFAR-10 and CIFAR-100 (Krizhevsky et al., 2014) with the ResNet Model
(He et al., 2016), and the General Language Understanding Evaluation (GLUE) tasks (Wang et al., 2019) with the BERT model (Devlin et al., 2019).
4. E XPERIMENTS ON I MAGE C LASSIFICATION T ASKS .
Data.
CIFAR (Krizhevsky et al., 2014) is a benchmark data set for image classiﬁcation.
We use both
CIFAR-10 and CIFAR-100 in our experiments, which are colorful images with 50000 training samples and 10000 validation samples, but from 10 and 100 object classes, respectively.
Setup.
The model we used is ResNet (He et al., 2016) with different depths.
We use random crop and horizontal ﬂip (Krizhevsky et al., 2012) to augment the original training images.
Since these operations do not change the augmented sample label, we directly adopt the original training sample label for all its augmented samples.
Following (He et al., 2016), we use the SGD with momentum optimizer to train each model for 200 epochs.
The learning rate starts from 0.1 and decays by a factor of 0.2 at epochs 60, 120 and 160.
The batch size is 128, and weight decay is 5e-4.
For each x i , | B ( x i ) | = 10 .
The λ P of the KL regularization coefﬁcient is 1.0 for both MMEL-H and MMEL-S.
The λ T in equation (8) for MMEL-S is selected from {0.5, 1.0, 2.0}.
We compare our proposed MMEL with conventional training with data augmentation (abbreviated as “Baseline(DA)”) under the same number of epochs.
Though MMEL can be computed efﬁciently in parallel, the proposed MMEL encounters | B ( x i ) | = 10 times more training data.
For fair comparison, we also compare with two other baselines that also use 10 times more data: (i) naive training with data augmentation but with 10 times more training epochs compared with MMEL (abbreviated as “Baseline(DA+Long)”).
In this case, the learning rate accordingly decays at epochs 600, 1200 and 1600; (ii) training with data augmentation under the framework of MMEL but with uniform weights on the augmented samples (abbreviated as “Baseline(DA+UNI)”).
Main Results.
The results are shown in Table 1.
As can be seen, for both CIFAR-10 and
CIFAR-100 , MMEL-H and MMEL-S signiﬁcantly outperform the Baseline(DA), with over 0.
points higher accuracy on all four architectures.
Compared to Baseline(DA+Long), the proposed
MMEL-H and MMEL-S also have comparable or better performance, while being much more efﬁcient in training.
This is because our backward pass only computes the gradient of the weighted loss instead of the separate loss of each example.
Compared to Baseline(DA+UNI) which has the same computational cost as MMEL-H and MMEL-S, the proposed methods also have better performance.
This indicates the efﬁcacy of the proposed maximal expected loss based reweighting strategy.
[Table]
Varying the Number of Augmented Samples.
One hyperparameter of the proposed method is the number of augmented samples | B ( x i ) | .
In Table 2, we evaluate the effect of | B ( x i ) | on the CIFAR data set.
We vary | B ( x i ) | in { 2 , 5 , 10 , 20 } for both MMEL-H and MMEL-S with other settings unchanged.
As can be seen, the performance of MMEL improves with more augmented samples for small | B ( x i ) | .
However, the performance gain begins to saturate when | B ( x i ) | reaches 5 or 10 for some cases.
Since a larger | B ( x i ) | also brings more training cost, we should choose a proper number of augmented samples rather than continually increasing it
[Table] [Table]
ESULTS ON
[Table]
ASKS
Data.
GLUE is a benchmark containing various natural language understanding tasks, including textual entailment ( RTE and MNLI ), question answering ( QNLI ), similarity and paraphrase ( MRPC , QQP ,
STS-B ), sentiment analysis ( SST-2 ) and linguistic acceptability ( CoLA ).
Among them, STS-B is a regression task, CoLA and SST-2 are single sentence classiﬁcation tasks, while the rest are sentence-pair classiﬁcation tasks.
Following (Devlin et al., 2019), for the development set, we report
Spearman correlation for STS-B , Matthews correlation for CoLA and accuracy for the other tasks.
For the test set for QQP and MRPC , we report “F1”.
Setup.
The backbone model is BERT BASE (Devlin et al., 2019).
We use the method in Section 3. to generate augmented samples.
For the problem of mismatching label as described in Section 3.3, we use a BERT BASE model ﬁne-tuned on the downstream task as teacher model to predict the label of each generated sample z in B ( x i ) .
For each x i , | B ( x i ) | = 5 .
The fraction of masked tokens for each sentence is 0.4.
The λ P of the KL regularization coefﬁcient is 1.0 for both MMEL-H and
MMEL-S. The λ T in equation (8) for MMEL-S is 1.0.
The other detailed hyperparameters in training can be found in Appendix C.
The derivation of MMEL in Section 3 is based on the classiﬁcation task, while STS-B is a regression task.
Hence, we generalize our loss function accordingly for regression tasks as follows.
For the hard loss in equation (7), we directly replace y z ∈ R with the prediction of teacher model on z .
For the soft loss (8), for each entry of f θ ( x i ) in loss term λ T (cid:80) z ∈ B ( x i ); z (cid:54) =
x i P ∗ θ ( z | x i ) MSE ( f θ ( z ) , f θ ( x i )) , we replace it with the prediction of teacher model if the difference between them is larger than 0.5.
Similar to Section 4.1, We compare with three baselines.
However, we change the ﬁrst baseline to naive training without data augmentation (abbreviated as “Baseline”) since data augmentation is not used by default in NLP tasks.
The other two baselines are similar to those in Section 4.1: (i) “Baseline(DA+Long)” which ﬁne-tunes BERT with data augmentation with the same batch size; and (ii)“Baseline(DA+UNI)” which ﬁne-tunes BERT with augmented samples by using average loss.
We also compare with another recent data augmentation technique SMART (Jiang et al., 2020b).
Main Results.
The development and test set results on the GLUE benchmark are shown in Table 3.
The development set results for the BERT baseline are from our re-implementation, which is comparable or better than the reported results in the original paper (Devlin et al., 2019).
The results for SMART are taken from (Jiang et al., 2020b), and there are no test set results in (Jiang et al., 2020b).
As can be seen, data augmentation signiﬁcantly improves the generalization of GLUE tasks.
Compared to the baseline without data augmentation (Baseline), MMEL-H or MMEL-S consistently achieves better performance, especially on small data sets like CoLA and RTE .
Similar to the observation in the image classiﬁcation task in Section 4.1, the proposed MMEL-H and MMEL-S are more efﬁcient and have better performance than Baseline(DA+Long).
MMEL-H and MMEL-S
[Table]
[Figure]
also outperform Baseline(DA+UNI), indicating the superiority of using the proposed reweighting strategy.
In addition, our proposed method also beats SMART in both accuracy and efﬁciency because they use PGD-k (Madry et al., 2018) to construct adversarial augmented samples which requires nearly k times more training cost.
Figure 2 shows the development set accuracy across over the training procedure.
As can be seen, training with MMEL-H or MMEL-S converges faster and has better accuracy except SST-2 and RTE where the performance is similar.
Effect of Predicted Labels.
For the augmented samples from same origin, we use a ﬁne-tuned task-speciﬁc BERT BASE teacher model to predict their labels as mentioned in Section 3.3 to handle the problem of mismatching label.
In Table 4, we show the comparison between using the label of the original sample and using predicted labels.
As can be seen, using the predicted label signiﬁcantly improves the performance.
By comparing with the results in Table 3, using the label of the original sample even hurts the performance.
[Table]
C ONCLUSION
In this work
[Table]
we propose to minimize a reweighted loss over the augmented samples to that directly considers the implicit their impacts on the loss.
Since we can not access the optimal reweighting strategy, we propose to minimize the supremum of the loss under all reweighting strategies
[Table]
and give a closed-form solution of the optimal weights.
Our method can be applied on the top of any data image tasks that the proposed method improves the generalization performance of the model efﬁcient in training
A P ROOF OF T HEOREM
Proof.
For any given x i and B ( x i ) , we aim to ﬁnd P θ ( · | x i ) on B ( x i ) such that
[Equation]
Since the objective is convex, by Lagrange multiplier method, let
[Equation]

[Equation] [Equation]
From ∇ P θ L ( P θ , λ ) = ∇ λ L ( P θ , λ ) = 0 , for any pairs of z u , z v ∈ B ( x i ) , we have
[Equation]
y
[Equation] [Equation]
Hence we have
[Equation]
Summing over z v ∈ B ( x i ) , we have
[Equation]
The proof completes.
B G ENERATING A UGMENTED S AMPLES FOR T EXTUAL S AMPLES
In this section, we elaborate the procedure of generating augmented sentences using greedy-based and beam-based method for a sequence.
For each original input sentence, we randomly mask k tokens (which is obtained by rounding the product of masking ratio and length of the sequence to the nearest number)
and then we do a forward propagation of the BERT to predict the tokens in those masked positions using greedy search.
The detailed procedure is shown in Algorithm 2.
We also use beam search (Yang et al., 2018) to generate augmented data.
The details of beam search can be referred to (Yang et al., 2018).
For sentence-pair tasks, we treat the two sentences separately and generate augmented samples for each of them.
Algorithm
Generate the i th augmented
( z i [ p j ] | z i ) .
In the following, we vary the factors that may affect the quality of the generated augmented samples.
These factors include
The number of masked tokens, which equals the replacement proportion multiplied with the sentence length.
This affects the diversity of augmented samples, i.e., replacing a large proportion of tokens makes the augmented sample less similar to the original one. 2.
Treating the two sentences separately in sentence-pair tasks when generating augmented examples, or concatenate them as a single sentence; 3.
Different generation methods like greedy search (Algorithm 2) and beam search.
The results are shown in Table 5.
As can be seen, compared with Baseline without data augmentation,
MMEL-H and MMEL-S under all hyperparameter conﬁgurations have higher accuracy, showing the efﬁcacy of data augmentation and the proposed reweighting strategy.
There is no signiﬁcant difference in using greedy search or beam search to generate the augmented samples.
In this natural understanding task, training with augmented samples generated with proper larger replacement proportion (i
[Table]
e., larger diversity) has slightly better performance.
For sentence-pair tasks, treating the two sentences separately and generate augmented samples for each of them has slightly better performance.
In the experiments in Section 4.2, we use Greedy search, masking proportion 0.
[Table]
and generate augmented sentence for each sentence in sentence-pair tasks.
[Table]
C H YPERPARAMETERS FOR THE E XPERIMENT ON THE GLUE B ENCHMARK .
The optimizer we used is AdamW (Loshchilov & Hutter, 2018).
The hyperparameters of BERT BASE model are listed in Table
[Table] [Table]