W HAT M AKES B ETTER A UGMENTATION S TRATEGIES ? A UGMENT D IFFICULT BUT N OT TOO D IFFERENT
A BSTRACT
The practice of data augmentation has been extensively used to boost the performance of deep neural networks for various NLP tasks.
It is more effective when only a limited number of labeled samples is available, e.g. , low-data or classimbalanced regimes.
Most current augmentation techniques rely on parameter tuning or inherent randomness; hence, their effectiveness largely varies on the tasks.
To efﬁciently ﬁnd the best augmentation strategy for each task, learning data augmentation policy is a promising solution, but the question of what makes a good augmentation in NLP tasks and how to design the reward function for learning a good policy remains under-explored.
To answer this, we hypothesize that good data augmentation should construct more diverse and challenging samples for providing informative training signals, while avoiding the risk of losing the semantics of original samples.
Therefore, we design a novel reward function for updating the augmentation policy to construct difﬁcult but not too different samples (DND).
Particularly, we jointly optimize a data augmentation policy while training the model, to construct the augmented samples with low conﬁdence but a high semantic similarity with original ones.
In addition, we introduce a sample re-weighting scheme to focus on difﬁcult augmented samples after the original ones are learned conﬁdently for more effective learning from the augmented ones.
Our learning-based augmentation outperforms the recent state-of-the-art augmentation schemes on various text classiﬁcation tasks and GLUE benchmark by successfully discovering the effective augmentations for each task.
Remarkably, our method is more effective on the challenging low-data and class-imbalanced regimes, and the learned augmentation policy is well-transferable to the different tasks and models.
I NTRODUCTION
Deep neural networks (DNNs) have shown near human- or superhuman-level performances on various NLP benchmark tasks.
This success, however, crucially relies on the availability of large labeled datasets, which typically require a lot of human efforts to be constructed (Brown et al., 2020; Cheng et al., 2020).
Although recent advance in language models (LMs) signiﬁcantly boosts state-of-the-art performances using self-supervised pre-training with a massive unlabeled dataset (Devlin et al., 2018; Liu et al., 2019), the number of given labeled samples on each downstream task is still critical for the performance and stability of ﬁne-tuned LMs (Sun et al., 2019; Zhang et al., 2020b).
Data augmentation is one of the most effective ways to efﬁciently use the given labeled samples by enlarging the amount and diversity with label-preserving transformations, so it helps to improve the generalization of DNNs (Lim et al., 2019).
Hence, the practice of data augmentation has been extensively used to boost the performance of DNNs and LMs for various NLP tasks (Jiang et al., 2020;
Qu et al., 2021), and it is more effective when only a limited number of labeled samples is available, e.g. , low-data or class-imbalanced regimes (Xie et al., 2020;
Kim et al., 2020).
Most current augmentation techniques rely on parameter tuning (Zhu et al., 2020) or inherent randomness (Wei & Zou, 2019); hence, their effectiveness largely varies between the tasks (see
Figure 1(a)).
To efﬁciently ﬁnd the best augmentation strategy for each task, learning augmentation policy ( i.e., distribution of multiple augmentations) has been explored as a promising solution (Cubuk
[Figure]
R e l a t i v e
I m p r
o v e m en t ( % )
IMDBNews
Review50Clinc
SST5TREC
[Figure]
[Figure]
[Figure]
[Figure]
et al., 2019; Hataya et al., 2020).
Here, a key for successful policy learning is on the design of reward function used to update it, which implicates what a good augmentation is; hence, various rewards are studied especially under vision tasks, e.g., maximizing validation accuracy (Cubuk et al., 2019) or training loss (Zhang et al., 2020c).
However, we found that these existing reward functions would not be enough for learning effective augmentation policy in NLP tasks.
This is due to the different nature of NLP tasks from discrete input space or vulnerability of the semantics for a slight modiﬁcation (Wang et al., 2018; Garg & Ramakrishnan, 2020). For example, we found that the recent state-of-the-art augmentation learning approach used for image classiﬁcation is not effective for NLP tasks as it suffers from losing the semantics of original sentence in augmentations (see Figure 1(b)).
This motivates us to explore what criteria make a good augmentation, and how to design the reward function for policy to ﬁnd them in NLP tasks.
Contribution.
In this paper, we develop a simple yet effective augmentation scheme for NLP tasks, coined learning to augment D ifﬁcult, but N ot too D ifferent (DND).
We ﬁrst design a novel reward function for updating augmentation policy under the following intuition: an effective augmented sample should be more difﬁcult than the original sample for providing ‘informative’ training signals, while maintaining its semantic meaning not too different from the original for avoiding ‘wrong’ signals.
Speciﬁcally, we compose the reward function using two complementary ingredients from training model: (1) model’s training loss for the given downstream task (on augmented samples) and (2) semantic similarity (between augmented and original samples) measured by using the contextualized sentence embeddings from the training language model.
We further apply a sample-wise re-weighting scheme when updating the model and policy with the augmented samples, to learn them more effectively by incorporating the original sample’s learning status.
Speciﬁcally, we assign more weight on the augmented sample, which (1) its corresponding original sample is learned enough, i.e. , has high conﬁdence, and (2) the conﬁdence gap between original and augmented samples is large.
With the proposed reward function and re-weighting scheme, the augmentation policy is simultaneously optimized during the model ﬁne-tuning with a gradient-based optimization, by applying an efﬁcient continuous relaxation to the non-differentiable components within the augmentation policy ( e.g., sampling the augmentations from the policy).
We demonstrate the effectiveness of the proposed augmentation policy learning scheme on various text classiﬁcation datasets and GLUE benchmark (Wang et al., 2019) , where our method consistently improves over the recent state-of-the-art augmentation schemes by successfully discovering the effective augmentation methods for each task (see Figure 1(c)).
For example, on the six different text classiﬁcation datasets, DND exhibited 16.45% and 8.59% relative test error reduction on average, compared to the vanilla and the previous best augmentation method, respectively.
We also found that DND is more effective on the challenging low-resource and class-imbalanced regimes, and the learned augmentation policy can be easily transferable to the different tasks and models.
This implies the broad applicability of our augmentation scheme: for instance, it might substitute the cost of augmentation learning on new tasks by using the pre-trained augmentation policies from DND.
R ELATED W ORKS
Data augmentation in NLP tasks.
Recently, the interest in data augmentation has also increased on NLP tasks (Feng et al., 2021) and various data augmentation approaches have been proposed,
[Table]
/qUrQQkBcRK3qR4SdpRzes668kSqd3m2lqq/K6Vk5dxJtDE+5C7pgs2f1zkLKgcF0yiYF2a+eIbJSGMbO9in+zxEEacooUzZHh7wiCftXIu1W+1uItVSiWcL34Z2/wnLope9</latexit> <latexit sha1_base64="fbUu5s5yMMH/NCJq+HICDZOxP2k=">AAACzXicjVHLTsJAFD3UF+ILdemmkZi4Iq0bXZK4caWYyCMiIW0ZYEJpm+nUhCBu/QG3+lvGP9C/8M44JCoxOk3bM+fec2buvX4S8lQ6zmvOWlhcWl7JrxbW1jc2t4rbO/U0zkTAakEcxqLpeykLecRqksuQNRPBvJEfsoY/PFXxxi0TKY+jKzlOWHvk9SPe44EnibrudSY3csCkN+0US07Z0cueB64BJZhVjYsvuEEXMQJkGIEhgiQcwkNKTwsuHCTEtTEhThDiOs4wRYG0GWUxyvCIHdK3T7uWYSPaK89UqwM6JaRXkNLGAWliyhOE1Wm2jmfaWbG/eU+0p7rbmP6+8RoRKzEg9i/dLPO/OlWLRA8nugZONSWaUdUFxiXTXVE3t79UJckhIU7hLsUF4UArZ322tSbVtaveejr+pjMVq/aByc3wrm5JA3Z/
[Figure]
[Figure]
[Figure]
such as word replacement with pre-deﬁned rules (Kang et al., 2018;
Wei & Zou, 2019) or pre-trained language models (Yi et al., 2021), back-translation (Xie et al., 2020), MixUp (Guo et al., 2019), and
Cutoff (Shen et al., 2020).
In a broader sense, adversarial training (Jiang et al., 2020; Zhu et al., 2020) can also be viewed as constructing augmented samples by adding perturbations to the word embedding.
Stacking of these augmentations has been shown to provide further improvement (Qu et al., 2021), but it requires the exhaustive searching cost for exploring each conﬁguration one by one.
Hence, the analysis what makes a good data augmentation is an emerging research direction (Miyato et al., 2018; Gontijo-Lopes et al., 2020).
For example, after the extensive study on the existing augmentations in the vision tasks, Gontijo, et al.
(Gontijo-Lopes et al., 2020) identify that the successful augmentations have the following properties: higher training loss and smaller distribution shift from the original.
But, any speciﬁc way to obtain such a desired augmentation is not suggested.
Learning-based data augmentation.
Automatically ﬁnding the effective data augmentation from data has naturally emerged to overcome the exhausting search cost from deciding which augmentation would be applied and tuning its parameters.
Learning augmentation policy ( i.e., distribution of multiple augmentations), optimized to ﬁnd the effective augmentation given a huge search space, is the representative method for this.
Here, a key for successful policy learning is a design of reward function used to update it; hence, various objectives for learning augmentation policy have been proposed, especially in the computer vision.
Maximizing the validation accuracy is arguably one of the most natural and successful approaches (Cubuk et al., 2019;
Hu et al., 2019).
Also, matching the distributions of original and augmented samples is popularly used as a learning objective by considering data augmentation as the problem of ﬁlling the missing data points of training distribution (Tran et al., 2017; Hataya et al., 2020).
Recently, the augmentation policy optimized to increase the training loss outperforms previous approaches (Zhang et al., 2020c; Wu et al., 2020).
On the other hand, ﬁnding the efﬁcient optimization method for learning augmentation policy is also widely explored to overcome the non-differentiability within the augmentation policy ( e.g., sampling the augmentations from the policy).
Reinforcement learning has been the representative method (Cubuk et al., 2019; Zhang et al., 2020c), but more efﬁcient continuous relaxation has been recently proposed (Hataya et al., 2020; Li et al., 2020).
However, such learning-based data augmentation schemes have been under-explored for NLP tasks (Ren et al., 2021).
DND: AUGMENT D IFFICULT , BUT N OT TOO D IFFERENT 3. O VERVIEW AND P ROBLEM D ESCRIPTION
Overview.
In this section, we present our technique, coined learning to augment Difﬁcult, but Not too Different (DND).
Our main idea is optimizing the augmentation policy to construct a sample that (a) is more difﬁcult than the original sample for improving the generalization of the classiﬁer, yet (b) maintains the semantics of the original sample.
In Section 3.2, we elaborate the reward and the training objective for our augmentation policy.
Problem description.
We ﬁrst describe the problem setup of our interest under a text classiﬁcation scenario.
Let D denote the given training dataset consisting of tuples ( x , y ) ∈ D where x =
[ x 1 , . .
, x L ] and y are the sequence of input tokens and the target label, respectively.
Our goal is to train a classiﬁer f θ ( x ) , initialized with a pre-trained transformer-based language model ( e.g.,
BERT (Devlin et al., 2018)), to minimize the task-speciﬁc loss L task ( x , y ) such as a cross-entropy loss (cid:96) xe
( p ( x ) , y ) where p ( x ) =
Softmax (cid:0) f
θ ( x ) (cid:1) .
For improving the model’s generalization upon a (limited) training data D , one can construct the augmented sample (cid:101) x
= τ ( x ) by applying a labelpreserving transformation τ .
In contrast to existing works relying on parameter tuning or inherent randomness for ﬁnding effective τ , we consider optimizing the augmentation policy T φ where τ ∼ T φ to maximize a reward function
R .
3. R EWARD AND TRAINING OBJECTIVE
To train our data augmentation policy T φ , we train it to maximize the following reward function:
[Equation]
where (cid:101) x
= τ ( x ) , τ ∼ T φ .
R task and R sim are the newly proposed reward functions to (1) provide ‘informative’ learning signal to training classiﬁer and (2) keep the semantic meaning of augmented sample (cid:101) x to be similar to the original x , respectively.
Rewarding difﬁcult samples.
To provide informative training signals, we encourage the augment policy to construct a harder view of the original sample; hard samples are known to facilitate training for many tasks, e.g. , object detection (Shrivastava et al., 2016), curriculum learning (Bengio et al., 2009) and data augmentation (Zhang et al., 2020c; Wu et al., 2020).
To this end, we reward the policy for generating an augmented sample (cid:101) x
= τ ( x ) , τ ∼ T φ with high training loss, while down-weighing the case where the high training loss incurred from the original sample x , not the augmentation policy
T φ .
To this end, we update the augmentation policy T φ to maximize the following reward:
[Equation]
where p ( x ) =
Softmax (cid:0) f
θ ( x ) (cid:1) .
Intuitively, the re-weighting factor w ( (cid:101) x ) emphasizes the samples of which (a) the original sample is conﬁdently learned ( i.e. , large p y ( x ) ) and (b) the gap of hardness between the original and the augmented sample is large ( i.e., large p y ( x )
− p y
( (cid:101) x ) ).
Rewarding not too different samples.
However, without proper regularization, the reward R task may derive the augmentation policy to generate samples that are semantically meaningless, e.g. , out-of-distribution samples (Gong et al., 2020; Wei et al., 2020) or samples losing the characteristics of the target label y .
See Figure 1(b) and 5(b) for a demonstration of such a behavior.
Hence, for reducing such a potential risk, we further update the augmentation policy T φ to keep the semantic meaning of augmented sample (cid:101) x be similar to the original x .
To this end, we ﬁrst extract the semantics of sentence x from its contextualized embedding o =
[ o 1 , . .
, o L ] where o (cid:96) is (cid:96) -th output of the transformer model (Reimers et al., 2019; Zhang et al., 2020a).
Then, a linear classiﬁer g W is applied to classify whether the given embedding pair ( o 1 , o 2 ) comes from the same sample ( y pos ) or not ( y neg ), respectively.
With this classiﬁer, we formulate the reward, R sim , for updating policy to correctly classify the given ( x , (cid:101) x ) as positive, hence maintain the semantic of original x as follow:
[Equation]
where (cid:96) xe is the cross-entropy loss with p W ( m , (cid:101) m )
=
Softmax (cid:0) g W ( m , (cid:101) m ) (cid:1) and g W ( m 1 ,
[Equation]
To train this linear classiﬁer g W , we add the following loss for training the classiﬁer:
[Equation]
where n is the mean-pooled contextualized embedding of the other training sample except x itself.
Furthermore, since the quality of contextualized embedding o is critical for our algorithm, we additionally constrain each o (cid:96) to preserve the semantics of input tokens x (cid:96) .
To this end, we additionally train our classiﬁer with a word reconstruction loss L recon :
[Equation]
Input: Classiﬁer from a pre-trained language model f θ , linear classiﬁer g W , data augmentation policy T φ , training data D , model learning rate η θ , policy learning rate η φ , update frequency T p for each iteration t do
Draw a mini-batch B = { ( x i , y i )
Bi
=1 } from D
Construct augmented samples from policy ˜ B = { (˜ x i , y i )
Bi
=1 } where ˜ x i = τ ( x i ) , τ ∼ T φ
Update model to minimize L task ( x i , y i ) + w ( (cid:101) x i )
L task ( (cid:101) x i , y i )
+ L sim ( x i , (cid:101) x i ) + λ r L recon ( x i )
if t % T p = 0 then
Update policy to maximize R task (˜ x i , y i )
+ λ s R sim ( x i , ˜ x i ) end if end for where p V ( o ) =
Softmax ( o V (cid:62) ) and V is the word embeddings matrix within the transformer model.
L recon effectively preserves the information within output embeddings learned through a pre-training phase; hence, it could provide better output embeddings.
More discussions are given in Appendix F.
In overall, our training loss of the classiﬁer is as follow with a ﬁxed hyper-parameter λ r :
[Equation]
3. C ONTINUOUS RELAXATION FOR TRAINING THE AUGMENTATION POLICY
Here, we describe the details on how the augmentation policy is optimized to maximize the nondifferentiable reward function deﬁned in Section 3.2.
We ﬁrst note that T φ consists of T consecutive operations O 1 , . . .
, O T where the operations are used to sample the augmentation from a pool of K augmentations τ 1 , . . .
, τ K .
Here, each operation O t is parameterized as O t ( p t , µ t ) , where p =
[ p 1 , . .
. , p K ] , (cid:80)
k p k
= 1 is the probability to sample the augmentation from the pool, and µ = [ µ 1 , . .
. , µ K ] is the magnitudes used when each augmentation τ k is sampled, i.e., τ k ( · ; µ k ) ∼ O ( p , µ ) with a probability p k .
Hence, as the non-differentiabiliy from operations breaks the gradient ﬂow from the classiﬁer ( e.g., sampling the augmentation), it is non-trivial how to optimize the parameters of augmentation policy φ = { p t , µ t } Tt =1 .
To this end, we adopt a continuous relaxation for each operation following (Hataya et al., 2020; Li et al., 2020), hence makes it be differentiable with respect to two trainable parameters of policy: probability p and magnitude µ .
Here, the key idea is using 1) Gumbel-Softmax (Jang et al., 2017) instead of Categorical distribution for sampling with p , and 2) straight-through estimator for µ (Bengio et al., 2013).
Therefore, by using this continuous relaxation, the classiﬁer and the augmentation policy would be alternately trained in a fully differentiable manner.
More discussions about the used augmentation pool and details are in Section 4.1 and Appendix A.3.
E XPERIMENTS
In this section, we evaluate our algorithm using two important NLP tasks: (1) text classiﬁcation and (2) entailment tasks.
We ﬁrst describe the experimental setups in Section 4.1.
In Section 4.2, we present empirical evaluations on DND and other baseline augmentations under both classiﬁcation and entailment tasks.
Finally, in Section 4.3, we provide additional analysis on our algorithm regarding (a) ablation of each component, (b) transferability of learned policy, (c) behavior of the learned augmentation policy.
In Appendix D and E, additional qualitative and quantitative results are presented, respectively.
Also, more ablation studies are given in Appendix F and G.
4. E XPERIMENTAL S ETUPS
Datasets and tasks.
For the text classiﬁcation task, we use the following benchmark datasets: (1)
News20 (Lang, 1995), (2) Review50 (Chen & Liu, 2014), and (3) CLINC150 (Larson et al., 2019) for topic classiﬁcation, (4) IMDB (Maas et al., 2011) and
SST-5 (Socher et al., 2013) for sentiment classiﬁcation, and (6) TREC (Li & Roth, 2002) for question type classiﬁcation.
For datasets without given validation data, we use 10% of its training samples for the validation.
With these datasets, we
[Table]
construct the 5-shot and class-imbalanced variants to further verify the effectiveness of augmentation for more challenging scenarios.
We also validate the proposed method on GLUE benchmark (Wang et al., 2019), which is a collection of diverse natural language understanding tasks: 1 regression task (STS-B) and seven different classiﬁcation tasks (RTE, MRPC, CoLA, SST-2, QNLI, QQP, and
MNLI).
As different metrics are used for each task in GLUE benchmark, we specify the used metric under the name of each task in Table 3.
For the evaluation, the given development set is used.
Detailed descriptions of tasks and construction of 5-shot and class-imbalanced datasets are in Appendix A.1.
Baselines.
We compare our method with various baseline augmentations in NLP tasks.
We ﬁrst consider a na¨ıve ﬁne-tuning without data augmentation, denoted by ( a ) Vanilla, followed by various augmentation schemes; ( b ) EDA (Wei & Zou, 2019): synonym replacement, random insertion, random swap, and random deletion are randomly applied; ( c ) Adversarial (Jiang et al., 2020; Zhu et al., 2020): adversarial example is constructed by adding adversarial perturbation on the word embedding; ( d )
BERT-aug (Yi et al., 2021): input sentence is randomly masked, then ﬁlled with pre-trained BERT to construct new sample; ( e ) Cutoff (Shen et al., 2020): continuous span of word is masked out similar to
(Srivastava et al., 2014)
(Aghajanyan et al., noise are to word
, then consistency is between original and perturbed ): two samples are mixed on the word perform source target and target to source translations; ( i ) Back&Adv (Qu et al., 2021): composition of back-translation and adversarial perturbation, that was identiﬁed as the best way for using multiple augmentations (Qu et al., 2021).
Details on the implementations are presented in Appendix A.2.
Training details.
All the experiments are conducted by ﬁne-tuning RoBERTa-base (Liu et al., 2019) using Adam optimizer (Kingma & Ba, 2015) with a ﬁxed learning rate 1e-5 and the default hyperparameters of Adam.
For the text classiﬁcation tasks, the model is ﬁne-tuned using the speciﬁed augmentation method with batch size 8 for 15 epochs.
For GLUE benchmark task, we commonly use batch size 16, except RTE task with batch size 8 following (Aghajanyan et al., 2021).
Under each augmentation, the model is ﬁne-tuned for 20 epochs on small tasks (RTE, MRPC, STS-B, CoLA) or 10 epochs on larger tasks (SST-2, QNLI, QQP, MNLI).
In the case of DND, we optimize the augmentation policy using Adam optimizer.
We choose hyperparameters in our method from a ﬁxed set of candidates; the update frequency T p ∈ { 1 , 5 , 10 , 100 } , λ s ∈ { 0 . 1 , 0 . 5 , 1 . 0 , 2 . 0 } , and ( α, β ) ∈ { (0 , 0) , (0 . 5 , 0 . 5) } based on the validation set.
Intuitively, the smaller datasets are beneﬁcial for smaller T p and λ s .
Other hyperparameters are ﬁxed as λ r = 0 .and
policy’s learning rate γ φ as 1e-3.
We use a simple two-layer MLPs for W .
As the data augmentation pool, we include all the baseline augmentation except MixUp because it synthesizes a completely new sample by mixing two original samples.
Hence, it is not clear whether it is meaningful to keep the similarity between original and augmented samples, which is the key component of DND.
Also, to reduce the computational burden from the external models, we only update the probability to be sampled for the word-level augmentations (BERT-aug, Back-trans, and EDA) by generating augmented sentences before training following prior works (Wei & Zou, 2019;
Yi et al., 2021;
Qu et al., 2021).
With this augmentation pool, our augmentation policy is composed of T =consecutive operations.
Also, to construct more diverse augmentations, we simultaneously train four
[Table] [Table]
augmentation policies by randomly selecting one of them for each mini-batch.
More details can be found in Appendix A.3.
4. E XPERIMENTAL R ESULTS ON T EXT C LASSIFICATION AND GLUE B ENCHMARK
To verify the effectiveness of DND [Table] we compare DND with various data augmentation schemes in the scenario of ﬁne-tuning RoBERTa-base model for each downstream task.
Table 1 summarizes the experimental results on six text classiﬁcation tasks, along with the challenging low-resource scenarios ( i
[Table]
e., 5-shot).
On the original full datasets, DND consistently outperforms the baseline augmentations for all datasets.
To be speciﬁc, DND exhibits 16.45% relative test error reduction compared to the vanilla method in the average.
Furthermore, compared to the previous best augmentation method for each dataset, DND exhibits 8.59% relative test error reduction in the average.
In the case of challenging 5-shot datasets, our method again outperforms the strong baselines with a larger margin.
These results show that our method successfully provides ‘informative’ learning signals to the training classiﬁer regardless of the number of given labeled datasets.
Next, we evaluate DND on class-imbalanced datasets; more practical setups suffer from the limited training data.
Here, we consider one more natural baseline, ‘Re-sampling’, that equally samples each class in a mini-batch.
Then, it is combined with each augmentation by applying augmentation instead of using duplicated samples for re-sampling (for details, see Appendix A.1).
For the evaluation, we report two popular metrics for class-imbalance: the arithmetic mean (bACC) and geometric mean (GM) over class-wise accuracy.
In Table 2, one can observe that the gain from data augmentation is more signiﬁcant compared to the case of balanced datasets, as the lack of diversity in minority classes would be addressed by it.
However, our learning-based augmentation scheme further enlarges this
[Table] [Table]
gain by constructing the augmented samples specialized for improving minority classes, based on its task-wise adaptability.
Speciﬁcally, DND exhibits 20.33% and 9.12% relative test error reductions in the average compared to Re-sampling and the previous best augmentation method, respectively.
Additionally, we perform t-test on the results in Tables 1 and 2 to validate the statistical signiﬁcance of DND compared to other baselines.
Here, we observe that the average p-value is 0.050, which is generally known to indicate the statistical signiﬁcance of the results.
Furthermore, we evaluate DND on the GLUE benchmark, composed of 8 different tasks.
Tablesummarizes the corresponding results, and one can observe that DND consistently improves the
RoBERTa-base model for all tasks; furthermore, DND outperforms the strong baselines for most cases.
We remark that the existing augmentations are sometimes even harmful, as they can’t consider the characteristics of each task.
For example, the goal of CoLA task is classifying whether the given sentence is grammatically valid or not.
Hence, this task is much sensitive to linguistic change, and it results in the degradation with the word-level augmentation methods.
In contrast, our method automatically reﬂects such inherent characteristics of each task.
4. A DDITIONAL A NALYSES
Ablation study.
To verify the effectiveness of the proposed reward function, we compare the following augmentation methods: (1) Vanilla : no data augmentation; (2) Fixed : ﬁxed augmentation is applied throughout the ﬁne-tuning.
We choose the baseline augmentation that shows the best performance in Table 1; (3) Random : augmentation is randomly selected among the augmentation pool at each training step; (4) Difﬁcult : policy is updated to increase the training loss of the augmented sample like Zhang et al.
(2020c); (5) Not Different : policy is updated to increase the similarity between augmented and original samples; (6) Validation : policy is updated to maximize the validation accuracy directly like Auto-augment (Cubuk et al., 2019).
To this end, we adopt LDM (Hu et al., 2019), which adapts an off-the-shelf reward learning algorithm from RL for joint data manipulation learning and model training.
As LDM originally uses a different augmentation pool, we adapt it using the same augmentation pool with DND.
(7) DND (no w ) : policy is updated to increase both training loss and semantic similarity without re-weighting scheme in Eq. 1.
Here, we commonly use the extra losses (Eq. 4 and Eq. 5) at training for the fair comparison.
Effect of these losses is shown in Appendix F.
From Table 4, one can verify that the effectiveness of Difﬁcult is quite limited due to it easily diverges to construct too different examples as shown in Figure 1(b) and 5(b).
Also, the augmented samples from Not Different do not much improve the performance as the learning signals from these samples would be duplicated with the original samples.
However, when we properly combine these rewards, it successfully provides ‘informative’ learning signals to the classiﬁer ( DND (no w ) ), and it can be further improved by the proposed re-weighting component ( DND ).
Remarkably, it is observable that DND clearly outperforms Validation .
These results demonstrate that our reward design is more effective than maximizing the validation accuracy for learning the augmentations.
Furthermore, we [Figure] emphasize that such methods maximizing validation accuracy usually require more computations than training loss-based methods due to an inherent bi-level optimization (Zhang et al., 2020c; Hataya et al., 2020); for example, LDM is 3x slower than DND under the same update frequency.
Transferability across datasets and architectures.
Here, we provide additional experiments to verify the transferability of the learned policy with our method.
To this end, we ﬁrst save the status of the augmentation policy for each update during the training on the source data and then directly apply it to other tasks or models as dynamic augmentations.
From Table 5, one can verify that the transferred augmentation outperforms the baseline augmentations and is even competitive with the performance of the jointly updated policy.
Also, the learned policy could be transferred to the different models, such as BERT (Devlin et al., 2018) and ALBERT(Lan et al., 2019).
These results imply that the proposed augmentation scheme would be greatly beneﬁcial for successfully applying the data augmentation to the new task or model without the exhaustive search and update.
More analysis on learned policy.
In Figure 3, we further present more analysis on the learned policy.
Speciﬁcally, we focus on the parameters in the ﬁrst operation
O 1 .
More detailed results can be found in Appendix B.
We ﬁrst note that the probability p is dynamically changed at an early stage, which implies the exploration step is naturally occurred.
After enough iterations, most of the augmentations are converged, and few candidates are only actively updated.
In contrast, each augmentation’s magnitude µ is progressively updated and almost converged at the last training iterations.
This stability is actually the result of our reward function for similarity, as the policy quickly diverges without this (see Appendix B).
Interestingly, from successful guidance of the proposed reward, we observe that our augmentation policy could re-discover the previously known best composition of augmentations (which was found by exhaustive grid search).
Also, it can newly discover the task-speciﬁc augmentations as shown in Figure 3(c).
It reveals the reason for success from DND, along with the efﬁciency of this method.
C ONCLUSION
We propose DND, a simple and effective augmentation learning scheme for NLP tasks.
We design of a novel reward function for optimizing augmentation policy to (a) construct the difﬁcult augmented samples for providing ‘informative’ signals, while (b) preserving their semantic meaning to be not too different from the original for avoiding ‘wrong’ signals.
From the experiments, we ﬁnd our augmentation method to be much effective in challenging low-data and class-imbalanced regimes.
Also, the learned augmentation policy transfers well across various tasks and models.
As the augmentation policy learning for NLP tasks is under-explored in the literature, we expect our work to contribute to exploring this direction of research.
Furthermore, since the proposed concept of difﬁcult but not too different augmentation is task- and domain-agnostic, we do believe that DND would beneﬁt in other tasks (e.g., self-supervised learning) and domains (e.g., vision and graph) as well.
On the other hand, incorporating the pre-trained models for augmentation learning in different domains (e.g., SimCLR (Chen et al., 2020)) would be an interesting future direction, as the ﬁne-tuning of such pre-trained models have shown remarkable successes recently.
E THICS STATEMENT
Data augmentation is generally effective for improving the performance on the target task; hence it now becomes a de-facto standard for training DNNs, in various input domains (Cubuk et al., 2019;
Park et al., 2019; You et al., 2020).
However, it also has a potential risk to amplify the undesirable property of the model at the same time, e.g., gender bias (Bordia & Bowman, 2019), since the enlarged diversity by data augmentation is proportional to the size of used samples for augmentation again.
Since the practice of data augmentation is usually based on pre-deﬁned ﬁxed augmentations, it is hard to remove such inherency of undesirable properties explicitly.
In this respect, the learning-based augmentation scheme would be greatly beneﬁcial, as it can easily alleviate this risk by setting the proper objective functions with respect to the desired property (Ziegler et al., 2019).
For example, one could construct the ‘fair’ data augmentation scheme by jointly adding the reward for fairness and task accuracy, as we have explored in this work.
Hence, we believe our proposed learning-based augmentation scheme would be a step towards improving a such fairness aspect in data augmentation, especially for NLP task.
R EPRODUCIBILITY STATEMENT
We describe the implementation details of the method in Section 4.1 and Appendix A.3.
Also, we provide the details of the datasets and baselines in Appendix A.1 and A.2, respectively.
We also provide our code in the supplementary material.
All the used packages are along with the code.
In our experiments, we use a single GPU (NVIDIA TITAN Xp) and 8 CPU cores (Intel Xeon E5-2630 v4).
A CKNOWLEDGMENTS
A D ETAILS ON E XPERIMENTAL S ETUPS A. D ATASET
Text classiﬁcation task.
We have used the following 6 text classiﬁcation datasets in Section 4.
Here, we set the maximum length of the input sentence as 128 except IMDB dataset with 512.
In case of 5-shot setup, it is further limited to 256 following (Hu et al., 2019).
(1) News20 (Lang, 1995) is a collection of 18,846 newsgroup documents, partitioned (nearly) evenly across 20 different newsgroups.
Originally, it is given with 11,314 training samples and 7,532 test samples.
News20 dataset is ofﬁcially available at http://qwone.com/ ˜ jason/ 20Newsgroups/ .
(2) Review50 (Chen & Liu, ) consists of Amazon reviews of 50 classes of electrical products.
Each class has 1,000 reviews and it is divided into 700 and 300 reviews for training and test samples, respectively.
Review50 dataset is ofﬁcially available at https://github.com/ zhouyonglong/Mining-Topics-in-Documents
(3) CLINC150 (Larson et al., 2019) is a intent classiﬁcation (text classiﬁcation) dataset with 150 in- domain intent classes.
In our experiments, we use the small version of CLINC150 which is ofﬁcially provided at https:/ github
com
clinc
oos eval . Here, each of the 150 in domain intent classes is given with 50 training, 20 validation
and 30 test samples.
4) SST-5 (Socher et al., 2013) is a dataset for sentiment classiﬁcation on movie reviews , which are annotated with ﬁve labels
; very positive positive neutral
negative, or very negative In the given SST -5 dataset, there are 8,544 training, 1,101 validation, and 2,210 test samples.
We have used SST- dataset at https://pytorch.org/text/0.8.1/datasets.html# sst
(5) IMDB (Maas et al., 2011) is a dataset for binary sentiment classiﬁcation on movie reviews.
It is composed of 25,000 labeled training samples and 25,000 test samples.
We use IMDB dataset provided at https://pytorch.org/text/0.8.1/datasets.html#imdb .
(6) TREC (Li & Roth, 2002) is a dataset for classifying the six question types, whether the question is about person, location numeric
information
and etc. There are 5,452 training samples andtest samples in TREC dataset at https://pytorch.org/text/ 8.1/datasets.html# trec .
GLUE benchmark.
GLUE
(Wang et al., 2019) is composed of 8 different tasks for natural language understanding: MNLI, MRPC, STS-B, CoLA, SST-2, QNLI, QQP, and RTE.
Here, we also set the maximum length of the input sentence as 128 for each task, except CoLA with 64.
Here, we offer a brief summary for each task.
More detailed description for each task can be checked at https://gluebenchmark.com/tasks .
(1) MNLI is a binary entailment task which is composed of 393k training and 20k development samples.
Given a pair of sentences, the goal is to predict whether the sentence is an entailment, contradiction, or neutral with respect to another one.
(2) MRPC is a binary classiﬁcation task for deciding whether the given sentence pair is semantically equivalent or not.
It is composed of 3.7k training and 408 development samples.
(3) STS-B is a regression task that scores (1 to 5) the semantic similarity between two sentences given in a pair.
It is composed of 7k training and 1.5k development samples.
(4) CoLA is a binary single sentence classiﬁcation task, where the goal is to predict whether the given English sentence is linguistically valid or not.
It is composed of
8.5k training samples and 1k development samples.
(5) SST-2 is a binary single sentence classiﬁcation task about movie reviews with human labels of their sentiment ( positive or negative ).
It is composed of 67k training and 872 development samples.
(6) QNLI is a binary classiﬁcation task that decides whether the given (question, sentence) pair contains the correct answer or not.
It is composed of 108k training and 5.7k development samples.
(7) QQP is a binary classiﬁcation task where the goal is to determine if two questions in a given pair are semantically equivalent.
It is composed of 364k training and 40k development samples.
(8) RTE is a binary entailment task similar to MNLI, composed of 2.5k training and 276 development samples.
Construction of 5-shot and class-imbalanced datasets.
In Section 4, we evaluate our method on 5-shot (in Table 1) and class-imbalanced regimes (in Table 2) which are artiﬁcially constructed from the existing benchmark datasets for text classiﬁcation task.
In case of 5-shot datasets, we randomly select ﬁve samples per each class to construct the datasets.
In case of class-imbalanced datasets, we consider “synthetically long-tailed” variants of SST-2,
News20, and Review50 datasets in order to evaluate our algorithm under various levels of imbalance.
For constructing the class-imbalanced training dataset, we assume the ordered numbers of data in each class as N 1 ≥ · · · ≥ N K .
We use a single parameter γ imb ≥ 1 , called the imbalance ratio , to control the class-imbalance of the dataset: once γ imb and N 1 are given, we set N k = N 1 · γ − k − 1 K − 1 imb so that
N 1 = γ imb · N K as done by Cui et al.
Namely, larger γ imb indicates more imbalanced class distribution.
We use N 1 = 1000 for SST-10 and N 1 = 400 for News20 and Review50, respectively.
For γ imb , we use the speciﬁed numbers in Table 2.
Re-sampling with data augmentation :
Following (Kim et al., 2020), we combine each data augmentation with batch-wise re-sampling, to handle the class-imbalance problem.
Namely, in order to simulate the generation of N 1 − N k samples for any k = 1 , 2 , · · · , K , we perform the data augmentation with probability N 1 − N yi N, for all i in a given class-balanced mini-batch B = { ( x i , y i ) } .
A. B ASELINES
(1) EDA (Wei & Zou, 2019):
Our implementation of EDA is based on the ofﬁcial implementation by the author at https://https://github.com/jasonwei20 .
4 operations in EDA (synonym replacement, random insertion, random swap, random deletion) is randomly applied for each word with a probability p eda , which is selected among { 0 . 15 , 0 . 3 , 0 . 45 } .
(2) Adversarial (Zhu et al., 2020):
We choose hyper-parameter δ for the norm constraint of adversarial perturbation among { 0 . 1 , 0 . 2 , 0 . 3 } based on its performance on the validation dataset.
We construct the adversarial example with a single step update under (cid:96) 2 norm.
(3) BERT-aug
(Yi et al., 2021):
Each token is randomly masked with a probability p mask and the masked token is replaced by the most likely token based on the output of pre-trained BERT model ( https://github.com/makcedward/nlpaug ).
p mask is tuned among { 0 . 15 , 0 . 3 , 0 . 45 } .
(4) Cutoff (Shen et al., 2020):
We choose hyper-parameter α for cutoff ratio among { 0 . 1 , 0 . 2 , 0 . 3 } based on its performance on the validation dataset.
It is observable that the dataset with shorter length usually beneﬁt from the smaller value, and larger α is better for dataset with longer length.
We adopt the ofﬁcial code at https://github.com/dinghanshen/Cutoff .
(5) R3F (Aghajanyan et al., 2021):
We sample the noise (cid:15) from the uniform distribution U =
[0, 1e-5], then it is added to the word embedding to construct the augmented samples.
Unlike other augmentation, the consistency loss is used for training.
The ofﬁcial code is available at https: //github.com/pytorch/fairseq .
(6) MixUp (Guo et al., 2019):
We adopt wordMixUp that mix two different sentences on their word embeddings.
We use the hyper-parameter α mix of mixing policy among { 0 . 5 , 1 . 0 , 2 . 0 } , which decides the mixing ratio λ mix where λ mix ∼ Beta ( α mix , α mix ) .
(7) Back-trans (Xie et al., 2020):
We use the en-de and de-en single models trained on WMT19 and they are available at https://github.com/pytorch/fairseq .
We use the default setups: beam search with beam size 5, and keeping only top-1 hypothesis.
(8) Back&Adv (Qu et al., 2021): After Back-translation is applied to the original sample, adversarial perturbation is further added to its word embedding.
Here, we choose hyper-parameter δ for the norm constraint of adversarial perturbation among { 0 . 1 , 0 . 2 , 0 . 3 } based on its performance on the validation dataset.
A. DND
For learning data augmentation via proposed DND, we use the following data augmentation pool:
Cutoff (Shen et al., 2020), Back-translation (Xie et al., 2020), BERT-aug (Yi et al., 2021), Adversarial (Zhu et al., 2020), EDA (Wei & Zou, 2019) and random noise (used in R3F (Aghajanyan et al., 2021)).
Here, we exclude MixUp in the augmentation pool because it synthesizes a completely new sample by mixing two original samples.
Hence, it is not clear whether it is meaningful to keep the similarity between original and augmented samples, which is the key component of DND.
Except for MixUp, we think any other augmentations can be in our augmentation pool.
Also, although both
BERT-aug and EDA have their own magnitude, we do not update these magnitudes, as it incurs the computational burden from the additional inference of external model (e.g., BERT) for each training iteration.
Instead, we adopt the common practice in text augmentations (Wei & Zou, 2019;
Yi et al., 2021;
Qu et al., 2021), which generate the augmented sentences using the pre-determined magnitude before training the model, similar to the case of Back-trans.
Hence, we only optimize the probability parameters for those word-level augmentations (BERT-aug, Back-trans, and EDA).
With this augmentation pool, our data augmentation is composed with T = 2 consecutive augmentations by ﬁnding their optimal probability and magnitude p and µ , respectively.
Since the word-level augmentations (BERT-aug, Back-trans, and EDA) can’t be applied after the embedding-level augmentations are applied, we limit the search space of O 2 to the embedding-level augmentations.
Also, to construct more diverse augmentations, we simultaneously train 4 augmentation policies by randomly selecting the one of them for each mini-batch.
Also, we empirically observed that the training of augmentation policy can be improved by introducing the auxiliary operation (Hataya et al., 2020;
Li et al., 2020) that decides whether to apply the selected augmentation or not, after sampling the augmentation from p .
Hence, we adopt this operation for training augmentation policy, but it is not necessary for the augmentation learning.
Finally, we remark that other training losses for augmented samples, such as consistency loss (Jiang et al., 2020; Xie et al., 2020) or contrastive loss (Qu et al., 2021), can be used for training the model ( L task ) and the augmentation policy ( R task ), although we have used the most simplest cross-entropy loss, i.e. , L task =
(cid:96) xe , to focus on the effect of augmentation itself.
B M ORE A NALYSIS ON L EARNED A UGMENTATION P OLICY
In this section
[Figure]
we further provide the dynamics of learned augmentation policies with respect to probability p and magnitude µ under two reward functions: (1) Difﬁcult , which was deﬁned in
Section 4.3, and (2) Difﬁcult, but not too Different (Ours).
As we have partially shown in Section 4.3, our reward function successfully guides the policy to learn the effective augmentation with progressive and steady updates on the parameters (in Figure 4).
In contrast, as one can see in Figure 5, rewarding the augmentation policy just to maximize the training loss would mislead policy to generate samples that are semantically meaningless, e
[Figure]
g., out-of-distribution samples or samples losing the characteristics of the target label y .
Speciﬁcally, in Figure 5(b), one can actually see evidence for this; the size of adversarial perturbation becomes too large.
Hence, this empirical observation again shows the importance of proper reward function to update augmentation policy and the validity of the proposed components in DND (maximizing semantic similarity and re-weighting scheme)
[Figure]
[Figure]
C C OMPLEMENTARY R ESULTS ON GLUE B ENCHMARK
In this section [Table] we provide the complementary results of GLUE benchmark (Wang et al., 2019). First, Table 6 shows the results on the development set including the standard deviation across 3 random seeds, have omitted for better presentation of the experimental results.
Next
[Table]
we also validate the effectiveness of ours method by comparing it with vanilla on the test set, which is available at https://gluebenchmark.com/submit .
As shown in Table 7, ours improves the performance of vanilla classiﬁer in most cases.
[Table]
[Table]
D M ORE Q UALITATIVE R ESULTS
Difﬁcult, but not different samples.
In this section, to further verify whether the augmented samples with the desired property are constructed from the learned policy, we construct the augmented samples using each augmentation method and compare the average difﬁculty and similarity in Figure 6(a).
We observe that the augmented samples from our augmentation policy actually have a desired property: they show signiﬁcantly low conﬁdence, i.e. , more difﬁcult, while it moderately preserves the semantic of original samples.
Furthermore, we present the augmented samples from BERT-aug by varying the difﬁculty and similarity in Figure 6(b).
Here, one can verify that the difﬁculty is solely not enough as too many changes can alter its semantic.
Also, the similarity is solely not enough as it tends to construct too easy samples which can’t provide ‘informative’ signals to the training model.
However, by considering two aspects jointly, reasonable and effective augmented samples can be constructed.
More qualitative results are presented in Figure 8.
Also, the plot of augmented samples by each augmentation scheme is presented in Figure 7.
[Figure]
[Table]
[Figure]
[Table] [Figure] [Table]
[Figure]
[Table]
, s : 0.97)
Worked for about 5 months.
Turned it off, wouldn't come back on without shaking it.
Cheesy plastic.
But looking bright when it did work.
[Figure]
[Figure]
[Table]
He released two
[Table]
these
[Table]
[Figure]
E M ORE Q UANTITATIVE R ESULTS
In this section, we present more quantitative results about the proposed DND.
Here, all experiments are conducted on News20 (Lang, 1995) dataset.
Also, all the values and error bars are mean and standard deviation across 3 random seeds.
The best result is indicated in bold.
[Table]
Compatibility of DND about model choice.
In Section 4, we mainly conduct experiments with
RoBERTa-base(Liu et al., 2019) model.
Hence, to verify the compatibility of DND about model choice, we further provide the experimental results under the different choice of pre-trained language models: RoBERTa-large (Liu et al., 2019), BERT-base (Devlin et al., 2018), and ALBERT-base (Lan et al., 2019).
RoBERTa-large has 24 Transformer blocks with 1024 hidden size and 16 self-attention heads (total parameters=340M).
BERT-base has the same architecture with RoBERTa-base (12 blocks with 768 hidden size and 12 self-attention heads, hence 110M total parameters), but it’s pre-trained in a different way.
ALBERT-base had an identical architecture with BERT-base except 128 hidden size (total parameters=12M).
In overall, the model size has following order: RoBERTa-large >
RoBERTa-base = BERT-base > ALBERT-base.
In Table 8, we compare DND with the followingaugmentations that are top-3 baselines in Table 1: Adversarial (Zhu et al., 2020), Cutoff (Shen et al., 2020), and R3F (Aghajanyan et al., 2021).
Here, DND outperforms all baselines and it shows that
DND successfully learns the effective data augmentation regardless of the model choice.
[Figure] [Figure]
[Table]
/u14tRl7GBTWzTPHexj0M00DT1P+ART9a5dWPdWncfqVap0Kzh27Lu3wGZLJlQ</latexit> <latexit sha1_base64="of2UylmpCSOaJF5f8w3Hmgycdts=">AAAC3HicjVHLSsNAFD2Nr/quunDhJlgEVyWRgi4FNy5cVLCt0GqZTEcbmheTiSChO3fi1h9wq98j/oH+hXfGCGoRnZDkzLn3nLl3rpcEfqoc56VkTUxOTc+UZ+fmFxaXlisrq600ziQXTR4HsTz1WCoCPxJN5atAnCZSsNALRNsbHuh
+0rI1I+jE3WdiLOQXUb+hc+ZIqpXWe+GTA04C/KjUS/vKmVLweNo1KtUnZpjlj0O3AJUUaxGXHlGF33E4MgQQiCCIhyAIaWnAxcOEuLOkBMnCfkmLjDCHGkzyhKUwYgd0veSdp2CjWivPVOj5nRKQK8kpY0t0sSUJwnr02wTz4yzZn/zzo2nru2a/l7hFRKrMCD2L91n5n91uheFC+yZHnzqKTGM7o4XLpm5FV25/aUrRQ4JcRr3KS4Jc6P8vGfbaFLTu75bZuKvJlOzes+L3AxvukoasPtznOOgtVNznZp7XK/u14tRl7GBTWzTPHexj0M00DT1P+ART9a5dWPdWncfqVap0Kzh27Lu3wGZLJlQ</latexit>
Here, we only report the results during one epoch for better visualization.
F D ETAILED A NALYSIS ON E XTRA L OSSES
In this section, we provide the detailed analysis on two extra losses L sim (Eq. 4) and L recon (Eq. 5) used when training the model, especially to investigate the following:
• For L sim (and R sim ), using g W with MLP is better than simpler choices such as cosine- similarity or (cid:96) 2 distance?
• What is the contribution of L recon in DND?
We remark that L sim is introduced to train the linear classiﬁer g W , which is used for rewarding not too different samples with R sim
(Eq. 2).
L recon is introduced to preserve the original semantics of input token, as the quality of contextualized output embedding is critical for our algorithm.
Here, all experiments are conducted on News20 (Lang, 1995) and Review50 (Chen & Liu, 2014) datasets.
Also, all the values and error bars are mean and standard deviation across three random seeds.
The best is indicated in bold.
Discussions with simpler choices for similarity loss.
Although other simpler choices, such as cosine-similarity or (cid:96) 2 distance on the output embeddings, are considerable as the similarity loss, using MLP allows a better trade-off between the original task performance and learning the similarity between the embeddings.
Namely, the simpler choices could over-regularize the output embedding and result in an underﬁtting for the target task.
However, by leveraging MLP, more information for the target task can be formed and maintained in the output embeddings.
Hence, g W with MLP can get properly trained during the ﬁne-tuning for the target task, even with a small number of training iterations.
For example, on Review50 dataset, we empirically observed that g W achieves 98.1% accuracy only after one epoch of training, as shown in Figure 9.
To empirically validate this intuition, we conduct the following experiments with DND using 1) cosine similarity and 2) (cid:96)distance between m 1 and m 2 , instead of using MLP.
As shown in the Table 9, the performance with the simpler choices actually decreases compared to using MLP.
We also remark that the effectiveness of MLP for measuring the similarity on the embeddings has been demonstrated in the other context as well, e.g., SimCLR
(Chen et al., 2020).
[Table]
More intuition and effect of reconstruction loss.
First, we present a more detailed description how reconstruction loss works.
Input token x ∈ { 0 , 1 } V is a V -dimensional one-hot vector with a vocabulary size V .
V ∈ R V × d is the word embeddings matrix of pre-trained transformer model that maps x to d -dimensional input embeddings z ∈ R d .
This z is feed-forwarded to the remaining parts of the transformer (e.g., self-attention blocks) and the corresponding output embeddings is o ∈ R d .
Hence, the resulting word reconstruction loss can be formulated as (cid:96) xe
( p V ( o ) , x ) (Eq. 5) where p V ( o ) =
Softmax ( o V (cid:62) ) .
Namely, it is predicting one-hot label (given by input token x ) from the dot product of output embeddings o and word embedding matrix V .
Intuitively, reconstruction loss L recon effectively preserves the information within output embeddings learned through the pre-training phase; hence, it could provide a better output embedding.
We empirically observed that the word reconstruction (i.e., prediction) from output embeddings becomes impossible as the model is ﬁne-tuned in the target task (See Figure 10).
We remark that this word reconstruction was almost perfectly possible before the ﬁne-tuning, as the language model is pretrained using a similar reconstruction loss (e.g., masked token prediction in BERT).
Thus, this observation implies that this semantic information is lost.
Our reconstruction loss effectively prevents such information loss and improves the quality of output embeddings.
To empirically demonstrate the effectiveness of this term, we further provide the experimental results of DND without reconstruction loss (i.e., λ r = 0 ).
As shown in Table 10, the reconstruction loss clearly improves the performance of
DND by providing better output embeddings.
[Table]
Effect of extra losses.
While the extra losses L sim (Eq. 4) and L recon (Eq. 5) are introduced for updating policy with DND, these losses could contribute to the learning of better representations and lead to the improvement themselves.
To verify this, we conduct the additional experiments about these auxiliary losses.
Here, when each loss is applied, the ﬁxed coefﬁcients are used: 1 for
L sim and 0 .
for L recon .
In Table 11, it is veriﬁed that each of the proposed losses could improve the performance contributing to the learning of better representations, and two auxiliary losses are complementary to each other.
[Table]
G M ORE A BLATION S TUDY
In this section, we provide more ablation study on the design choices of DND.
Here, all experiments are conducted on News20 (Lang, 1995) and Review50 (Chen & Liu, 2014) datasets.
Also, the same values are used for the others except for the speciﬁed hyper-parameters.
All the values and error bars are mean and standard deviation across three random seeds.
The results with originally used values are underlined.
Number of augmentation policies n T .
As we have mentioned in Appendix A.3, we simultaneously train multiple augmentation policies ( n T = 4 ).
To see the effect of this, we compare the cases with less ( n T = 1 ) and more ( n T = 10 ) number of policies in Table 12.
Overall, the improvement from
DND is not sensitive to the number of augmentation policies, but using multiple policies shows slightly better results as it would construct more diverse augmentations.
[Table]
Number of operations T .
As described in Section 4.1, our augmentation policy is composed of two consecutive operations.
Here, we compare this choice ( T = 2 ) with less ( T = 1 ) and more ( T = 3 , 4 ) number of operations.
As shown in Table 13, one can verify that a single operation is not enough to construct complexly augmented samples; hence the empirical gain is relatively small compared to the original choice.
With more operations, almost saturated improvement is observed.
Remarkably, similar tendencies are also observed in vision tasks (Hataya et al., 2020; Cubuk et al., 2020).
[Table]
Different values for α and β .
We additionally provide the results with different values for α and β , which are introduced for re-weighting scheme in R task (Eq. 1).
In Table
[Table]
it is observable that each component of the re-weighting scheme is solely effective, and the gain from re-weighting is not much sensitive to the choice of α and β .
We remark the original choice ( α, β ) =
(0 . 5 , 0 . 5) shows the best empirical gains
[Table] [Table]
Different values for λ r in the reconstruction loss.
We conduct additional experiments with different values of λ r although we use a ﬁxed value ( λ r = 0 . 05 ) for the reconstruction loss (Eq. 5) in all the experiments.
As shown in Table 15, the reconstruction loss with moderate values of λ r clearly improves the performance, but too large value decreases the performance as it excessively updates the model for reconstruction loss.
Hence, we recommend using the originally ﬁxed value ( λ r = 0 . 05 ).
[Table]
Update frequency T p and weight λ
s for policy T φ .
As described in Section 4.1, we mainly tune
T p and λ s for each task.
Table 16 and Table 17 show the results of different values of them.
Here, we observe the consistent improvement of DND with varied T p , which indicates an efﬁciency of DND.
In the case of λ s , the performance of DND still signiﬁcantly outperforms the other baselines with all λ s , but the gain is varied with the chosen λ s , as this hyper-parameter directly affects the trade-off between ‘informativeness’ and ‘risk’ of the augmented samples.
[Table] [Table]
Effect of auxiliary operation.
As we have mentioned in Appendix A.
[Table]
we adopt the auxiliary operation for our implementation, which has been originally used by Hataya et al.
Here, we provide the results without this auxiliary operation.
As one can see in Table 18, this operation is not signiﬁcant to the gain from DND.
[Table]