Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The proposed method DND learns an augmentation policy to augment a target text dataset. The augmentation policy is optimized to create difficult yet not too different examples from the original data. To further improve the performance, a sample re-weighting scheme is introduced to focus on harder examples. Experiments show that learned augmentation policy can achieve good results on various NLP tasks and transfer well across datasets and architectures.
main_review: Strengths:
-	__Clarity__. The writing is clear and easy to follow.
-	__Well-motivated approach with extensive experiments.__ Although the approaches to learn an augmentation policy using an adversarial objective, enforce the augmented samples to be similar to the original samples and use continuous relaxation to train the augmentation policy are studied separately in image domain, the proposed method is a viable way to apply the ideas to text data and shows good experimental results on various NLP tasks.

Weaknesses/comments:
-	__Comparison with other learning objectives.__ The ultimate goal of data augmentation is to improve the generalization power of a model. How does the proposed Difficult and Not Different objective compare with the objective that improves the validation performance directly like AutoAugment or TAA [1].
-	__Sensitivity of DND hyperparameters.__ Despite being an Automated Data Augmentation method, there are several important hyperparameters, e.g. $\lambda_r, \lambda_{sim}, \lambda_s, T_p$, to be tuned. How sensitive is the proposed method to these hyperparameters? Whether these hyperparameters needed to be tuned carefully?
-	__Ablation study for extra loss terms.__ In the ablation study section, are the ‘Vanilla’, ‘Random’ and ‘Fixed’ baselines also being trained with the extra loss terms $L_{sim}$ and $L_{recon}$? While it is a motivated decision to introduce these terms during the training of $f_\theta$, can these losses also contribute to the learning of better representations and lead to the improvement? It would be useful if the effects of these extra losses are discussed in the ablation study. 
-	__Exclusion of MixUp from the augmentation pool.__ Authors mention that MixUp is not included in the augmentation pool as it alters the semantics of the original sentences. However, MixUp can create difficult examples by sample and label mixing. MixUp also shows good results for some datasets under Table 1. As a learning-based data augmentation method, is it the responsibility for the search algorithm to learn the use of MixUp in a data-driven way? Can the inclusion of MixUp in the augmentation pool improves the performance? If the validity of a candidate augmentation method has to be evaluated before adding to the augmentation pool, does it contradict with the goal of fully learnable data augmentation?
-	__Hyperparameter tuning for other baselines.__ Some of the baseline methods, like EDA, BERT-Aug, MixUp, Back&Adv also involve hyperparameters. According to Appendix A2, it seems that these hyperparameters are not tuned. As the hyperparameters of DND (e.g. $\lambda_s$ and $T_p$) are tuned for each dataset, it is fairer to tune the hyperparameters for the other baselines. 
-	__Formulation of the probability and magnitude for EDA.__  Is there a specific reason that EDA is assigned with a single probability and magnitude parameter? Can the use of different $p$ and $\mu$ values for synonym replacement, random insertion, random swap, and random deletion further improves the augmentation policy?
-	__Definition of the magnitude parameters for the augmentation candidates.__ It is unclear how the magnitude parameters are defined for the augmentation candidates. For example, is the mask probability of BERT-Aug taken as the magnitude? What are the ranges of the magnitudes? It would be useful to include these in the appendix.

Reference

[1] Shuhuai Ren, Jinchao Zhang, Lei Li, Xu Sun, Jie Zhou. Text AutoAugment: Learning Compositional Augmentation Policy for Text Classification. _arXiv preprint arXiv:2109.00523,_ 2021
summary_of_the_review: Although the approach to learn an augmentation policy using the adversarial and similarity objectives is studied in the image domain, the proposed method contributes to the adaptation of learnable augmentation in the NLP domain and shows good experimental results. Overall, I tend to recommend for acceptance for the initial rating.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The authors proposed a new reward function to learn the distribution of augmentation policies for NLP tasks, which can generate "difficult" and semantic similar samples to facilitate training. They further introduced a sample-wise re-weighting scheme to leverage the learning status of original sample. A continuous relaxation is applied to optimize the trainable parameters in policy. The proposed method outperformed SOTA on text classification and entailment tasks. They also conducted extensive studies and analysis, demonstrating the effectiveness of their method on low-resource and class-imbalanced regimes, as well as its transferability.
main_review: Strengths:
The proposed method is intuitive, simple to apply and effective over many baseline augmentation methods on various datasets, especially on low-resource, class-imbalanced regimes. Besides the good performance on benchmarks, the authors also conducted good ablation study and analysis on learned policies, showing that the selected policy coincides with previous work (via exhaustive grid search).

Weakness and Concerns:
1. The main issue lies in the choice of baseline. As a learning-based augmentation method, it will be more convincing to compare with other learning-based method (using same pool), e.g. auto-augment, adversarial auto-augment etc. If the method can still outperform them, it can better demonstrate the effectiveness of their reward design.
2. Limited technical novelty: The whole idea is very straight-forward and most of the techniques are borrowed from previous works. 
summary_of_the_review: This paper proposed a simple and effective reward function for learning-based augmentation selection in NLP. Extensive experiments and analysis demonstrated its effectiveness on text classification and entailment, especially on low-resource and class-imbalanced regimes.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a data augmentation technique --- difficult but not too different. The authors claim that an augmented sample should be difficult (in terms of loss) and semantically similar to the original sample. The authors propose two rewards functions to reward difficult samples and similar samples, respectively. A relaxation technique is further used such that the augmentation policy can be trained. Experiments are conducted on the GLUE development set and several other text classification tasks. The proposed method improves upon existing ones.
main_review: Overall, the paper is well-written and easy to understand. Experiments are conducted on different datasets and training schemes. The proposed method improves upon existing data augmentation techniques.

The proposed algorithm in this paper is a combination of known techniques, i.e., all the data augmentation techniques are well-established, and the relaxation method used to train the policy is also not new. Because the technical novelty of this paper is limited (which is acceptable), empirical evaluation needs to be substantially strengthened.

Some designs are not well-motivated. 
1. In the similarity loss (Eq. 4), a two layer MLP is used to serve as a linear classifier. Could the authors explain why this is better than simpler choices such as cosine-similarity or $\ell_2$ distance?
2. Continue on Eq. 4, what is the accuracy of $g_W$ after training? It is possible that $g_W$ cannot get properly trained because of the limited number of fine-tuning epochs.
3. The reconstruction loss (Eq. 5) is confusing. Could the authors provide more intuition on this? Also, there should be experiments that demonstrate the necessity of this loss term.

The training looks ad-hoc. There are several designs that seem arbitrary. Some additional experiments are needed.
1. Several existing methods are selected as the data augmentation pool (Cutoff, Back-trans, BERT-aug, Adv, EDA and R3F). There should be experiments that show the importance of these policies, i.e., are they all necessary? In Figure 3a, the probability of selecting R3F, Adv, EDA and Cutoff is low (Cutoff nearly drops to 0) towards the end of training. Is it possible to only include Back-trans and BERT-aug?
2. The authors mention that they need to simultaneously train 4 augmentation policies. What will happen if we only train one (or more than 4) policy?
3. The operation count is set to $T=2$ in the experiments. Analysis are needed on other values of $T$.
4. The authors mention an auxiliary operation (Hataya et al., 2020) is added. An ablation study is needed regarding the significance of this operation.
5. In the sample weight $w$ (Eq. 1), there are two hyper-parameters $\alpha$ and $\beta$. Experiments are needed to see how these two hyper-parameters change model performance.
6. There should be experiments on the weight ($\lambda_r$) of the reconstruction loss (Algorithm 1).

Additional comments:
1. In Table 1, performance gain is not clear. Variance of the results is very large, and it is hard to tell the significance of the gain. The authors should conduct significance tests and report the p-values.
2. In the current version, the training loss function is only shown in the Algorithm box. Include it at the beginning of Section 3.2 will make the presentation clearer.

I will raise my score if the authors can conduct the ablation experiments and address my concerns.
summary_of_the_review: The paper applies existing differentiable data augmentation methods (Hataya et al., 2020) to text classification. Overall, the technical novelty is low. There are several additional experiments (in particular ablation studies) needed to justify the design choices and the performance gain.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper provides a simple yet effective approach towards data augmentation in NLP tasks. In particular, it proposes an augmentation strategy that encourages constructing augmented samples with low confidence (which makes them challenging, hence more informative) and high semantic similarity with the original samples (which ensures the semantics are not lost in augmentation). Through comprehensive experiments, the authors show that their approach outperforms recent state-of-the-art techniques, especially on low-data and class-imbalanced regimes.
main_review: ##########################################################################  
Strengths: 
- The paper is very well written. The motivations and contributions of the paper are very clear and important. Data augmentation policy learning is a relatively under-explored area in the literature, and I think this paper makes a very useful contribution to exploring this research area. The proposed approach is simple enough to be replicated easily in follow-up research. The provided code in the supplementary material is well written and easy to understand too.
- The paper provides comprehensive experiments covering a diverse set of tasks. It includes results using 8 baseline/state-of-the-art models on 6 datasets with different text classification tasks, and on 8 tasks present in the GLUE benchmark. It provides the results on multiple class-imbalance/low-data setups. Further, it provides an ablation study to understand the contribution of different components of the proposed model, to provide evidence of the importance of each of those components. Finally, the paper also provides some results to show the transferability of the approach, and some analysis of learning dynamics. In most of these results, the proposed approach outperforms the baseline/state-of-the-art models.

##########################################################################  
Weaknesses / Suggestions:
- Minor correction in page 4: "Rewarding not too different samples" section. It's mentioned to see "Figure 1(b) and 6(b)". But the figure 6 is not present in the main paper. Further, both "1(b)" and "6(b)" are linked to 1(b) figure only.
- `gW` from equation (3) is sent directly to the cross entropy loss function in equation (4). I think it would be more clear if a sigmoid function is added on `gW` either in equation (3) or (4). Further, in equation (3), did you consider simply calculating the dot product of `m1` and `m2`, instead of concatenating them and using a neural network?
- I think the equation (5) is a bit unclear. Could you clarify the following:
    - Is the idea here similar to minimizing the dot product between the output embedding and input embedding for a given word?
    - Can you add more description of what exactly `V` and `x` are? `x` is referred to as the input token, but its used as a label in the cross entropy loss, implying that it’s a binary value. How is the input token converted to a binary label?
    - Given that the output and input embeddings of a transformer are in different spaces, could you provide some justification of how calculating the dot product between those vectors would be appropriate?
- A minor comment, but the usage of policy/reward made me think that this paper uses some reinforcement learning approach. Upon reading the paper, the usage of those terms do seem appropriate. I am not suggesting that this needs a change, but I just wanted to point out that some readers might get confused a bit by this terminology.
- Regarding the GLUE benchmark results, it would be interesting to see how this model fares in the GLUE leaderboard. Could you add those results if available?
- In the final conclusion section, could you add some directions in which this work can be extended?
summary_of_the_review: Overall, I vote for accepting. I think the paper proposes a very interesting approach to improve data augmentation in NLP tasks by learning a policy that decides how to combine different augmentation techniques in a task dependent manner to generate samples that are informative while retaining the semantic information from the original sample. Since this research area is relatively new, I think this paper provides a really good baseline, and a framework for other papers to improve upon, as its easy to implement and allows adding more augmentation techniques in the policy pool, improving the loss/regularizer functions, etc. I have a few minor issues regarding the paper, and I described them in detail below. Hopefully the authors can address my concerns in the rebuttal period.
review_title: null

Meta-review: We appreciate the authors for addressing the comments raised by the reviewers during the discussion period, which includes providing more experimental results to address the concerns. We believe the publication of this paper can contribute to the important topic of data augmentation.

The authors are highly recommended to consider all the comments and suggestions made by the reviewers when further revising their paper for publication.
Decision: Accept (Poster)