Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: In this work, authors propose a text-generation method to augment training data for various discriminative few-shot NLP tasks (from the Glue benchmark) such that the model performs better on these tasks as opposed to training (e.g. prompt-tuning/fine-tuning) using the original train split alone. Towards this end, authors propose i) a discriminative loss to ensure generated texts produce enough diversity to differentiate between labels and ii) a meta-weighted generative loss to take into account the impact of different tokens towards a label. They also propose an ensembling classifier to add more robustness to deal with generated samples. Experimental results on various tasks from the GLUE benchmark shows that the proposed method outperforms other methods from the literature.
strength_and_weaknesses: **Strengths**
* The paper is enjoyable to read and the algorithm has been developed nicely throughout the narratives. The authors clearly describe the problem they are intending to solve followed by motivating and describing the three major components of their algorithm - the discriminative loss, meta-weighted generative loss and the ensembled classifier. Authors also provide the end-to-end algorithms which help the reader to understand and possibly implement the proposed method. 
* Experimental results on the GLUE benchmark show clear and non-trivial improvement over existing baselines including the ones that rely on generating additional samples and those which do not.
* Although the final proposed method is somewhat complicated, ablation studies on the importance of each of the component show relative improvement caused by individual components and the importance of all these pieces. 
Overall, I find it to be a strong paper with a non-trivial contribution, clear narrative and strong empirical results.

**Weaknesses/Comments**
* Qualitative analysis - Given that it is a generative paper, I'd like to see examples of how the generated texts look like for different labels. How different are those from the ones present in the training sample? On the same line, I'd like to see how well the meta-weight generation can learn the importance of various tokens. For example, authors provide an example that the token "movie" might not be relevant to generate texts for positive/negative movie reviews. Does such properties automatically emerge from the meta-learning procedure?
* Susceptibility to hyper-parameter tuning - The optimization procedure is performing three interleaved gradient updates at every iteration. I'd imagine that such an optimization procedure will require sophisticated hyper-parameter tuning to obtain stable convergence. I'd like authors to comment on that along with the overall computational cost for this procedure.
* The whole generator-discriminator paradigm is very similar to many works from the literature which are based on GANs. I'd like the authors to clarify in the paper how their optimization process varies differently from the GAN.
* Generalizability of the ensembled classifier - Can the ensembled classifier generalize to other methods which also produce synthetic samples to augment training set? If so, how much gain can we tentatively expect to obtain with those methods? 

clarity,_quality,_novelty_and_reproducibility: **Clarity**
The narrative of the manuscript is clear and pleasure to read - the problem is clearly described and every piece of the overall proposed method is well motivated and explained. 

**Quality**
The proposed method looks sound to me and it intuitively makes sense. Experimental results and ablation studies confirm the validity of the proposed method. As mentioned above, adding more qualitative evaluations will make the contribution stronger.

**Novelty**
Although individual components might not be sufficiently novel, the overall contribution (i.e. the end-to-end algorithm) is novel and it cleverly combines and modifies existing method to solve the desired problem. I have no concerns regarding the novelty.

**Reproducibility**
Authors provide sufficient details and pseudo-code corresponding to their methods. As mentioned above, some additional details on the stability and ease-of-training of the meta-learning procedure will be helpful (apologies if I have missed it).
summary_of_the_review: Overall, I found the paper to have a solid technical contribution to solve the task at hand which is to improve few-shot performance of various discriminative NLP tasks by learning to generate diverse synthetic samples. Experimental results and ablation studies showcase the efficacy of the algorithm. At this point, I am inclined to accept this paper based on my opinion alone.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The paper proposes a method to augment label-discriminative data in PLM few-shot learning setting. The main idea is to improve the PLM-generator by introducing a label-discriminative loss in a meta-learning framework. The improved PLM-generator is then used to generate augmentation data to finetune a classification PLM(Roberta-large). The paper compares the proposed methods with several few-shot baselines on seven classification tasks of GLUE.

strength_and_weaknesses: Strength:
1. The overall framework to investigate the few-shot learning ability of PLM  is somehow novel.
2. It is interesting to design the $L_{disc}$ to encourage the PLM-generator to synthesize more label-discriminative augmentation examples.
3. The paper is well-written with clear logic. Most part of the paper is easy to understand.


Weakness:
1. The intuition that formulates label-discriminative text generation into a bi-level problem is not clear. What is the interpretation behind it? It seems that the bi-level structure is not necessary in this case, since there is no strict hierarchy of the inner and outer objectives ($L_\text{gen}$ and $L_\text{disc}$): the order of inner and outer objectives can be simply swapped to achieve the same goal(i.e. using unweighted generation loss in the outer loop and weighted discriminative loss in the inner loop). The bi-level form here is more like to introduce “technical contribution”.
2. The two reason for not directly combining $L_\text{disc}$ and $L_\text{gen}$ is not well justified ( "the generation-irrelevant loss  $L_\text{disc}$  will unavoidably interfere with the language modeling process” and “A hyperparameter needs to be introduced to balance the weights of the two losses ”.) I do not see why the bi-level formulation helps solve these issues. Specifically, suppose the token weights are learned to minimize the discriminative loss, how can it guarantee such weights will not affect the language modeling process? In fact, even without direct intervention, the $L_\text{disc}$ will affect $w$ and therefore affect the optimization of $L_\text{gen}$. Besides, it seems that introducing a hyperparameter to balance  $L_\text{disc}$  and $L_\text{gen}$ is acceptable, as the author introduces such a hyperparameter in Eqn. 5 to balance the two loss term for finetuning, which is actually in a similar situation.
3. I remain skeptical about the performance gain of the proposed method. Firstly, the authors do not compare with the important baseline that directly uses few shots in the demonstration(as in-context examples) to generate augmentation data, and then use the labeled few shots and synthetic data to fine-tune a PLM classifier. I find that the cited paper (Meng et al., 2022) uses the same PLM generator and classifier, as well as a much restricted zero-shot setting, which can achieve comparable performance with FewGen in some task. It looks like the performance gain (if any) is quite marginal, given that few-shot should already introduce some performance boost compared with zero-shot. Besides, I am curious about to which extent more label-discriminative data can improve the moderate-size PLM classifier. Table2 should include all the glue tasks. Complete results of the important baseline(w. $L_\text{gen}$) can help justify which margin of improvement can the proposed method achieve. 
4. The paper does not conduct quantitative and qualitative analysis to measure the quality of the generated text. Can the improved generator really generate more label-discriminative text? Some automatic metrics/human evaluation should be introduced to measure the quality of the generated data. 
5. Usually, the bi-level formulations used in previous work present justifications of why such formulation is adopted, such as meta weight net, MAML, etc. However, no theoretical justification or intuitive explanation is given to explain the necessity and superiority of such a formulation in this paper.
6. The performance bottleneck of the proposed method is quite obvious, since it is not applicable to large PLMs, and moderate PLMs(e.g. 1.6B Crtl) usually do not have good generation ability. From the experimental details, it is quite concerning that for the experiments using Ctrl as PLM-generator, the batch size can be only set to 2 on A100 GPUs. This brings the usability of this method on larger PLM into question. Besides, it is well known that meta-learning methods are notoriously hard to tune and can be extremely sensitive to hyperparameter choices. It is hard to tune the framework on a large PLM with a small batch size.  In addition, the updating process of the generator looks time-consuming, as the gradient calculation for prefix tuning needs to backward pass all the gradients through the PLM’s parameters in each iteration.
7. The authors only conduct experiments on 2-class or 3-class NLU classification tasks. It is questionable whether the proposed method is suitable for other tasks(e.g., multi-class classification and QA tasks). 

clarity,_quality,_novelty_and_reproducibility: Clarity, Quality, and Novelty: The paper is well-written and easy to understand. The intuition that formulates label-discriminative text generation into a meta-learning framework is not clear and a bit sudden.

Reproducibility: Code is provided in supplementary materials.



summary_of_the_review: Overall, the paper is well-written with a clear flow. However, (1) the intuition that formulates label-discriminative text generation into a bi-level problem is not clear, as there is no strict hierarchy of the two objectives. No convincing intuitive explanation or theoretical justification is given to explain the necessity and superiority of such a meta-framework in this paper. (2) Some important baselines are missing, and I remain skeptical about the quality improvement of generated data and performance gain of the proposed method.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes a new augmentation strategy to augment few-shot samples during fine-tuning. The core idea of the paper is to use a pre-trained language model (PLM) to generate texts which can be leveraged towards fine-tuning.  The authors propose two methods in the paper to achieve this: (i) a method to fine-tune the PLM to generate appropriate texts; (ii) a technique to fine-tune the classifier using the generated texts.
strength_and_weaknesses: Strengths:

	- The paper shows strong improvements over other baseline few-shot fine-tuning techniques (with / without use of data-augmentation).  The comparisons with data-augmentation techniques is more fair, as extra data is used during downstream fine-tuning.

	- The main technical novelty is in formulating the fine-tuning process of the generator with the meta-objective. Conditioning the generator on a weighted version of the input is a good way to generate label-coherent examples and the authors make the framework to work which is a strength. 

Weakness:

	- Could the authors add the ablation where D_train and D_gen is together used for fine-tuning?  I could not find it in the paper and it would be beneficial to add this to truly understand the effectiveness of the proposed fine-tuning strategy (with temporal ensembles) for the classifier tuning stage. In Table. (2), I do find -temporal ensemble, however I am assuming it is a two step fine-tuning - first on D_train and then on D_gen without the regularizer. 

	- The compared baselines are not comprehensive. The authors should add more baselines where data-augmentation is used for few-shot learning. Some of the baseline techniques from https://aclanthology.org/2021.findings-acl.84.pdf can be investigated to strengthen the contribution of the paper. While a comprehensive analysis of all the augmentations is out of scope for the paper, a few strongly performing augmentation techniques can be added as baselines to Table 1.

I would like to see some quantitative analysis on what proportion of label noise is mitigated with the use of meta-objective in training the generator? Downstream performance is definitely one indirect way to measure it, but having a quantitative analysis would strengthen the paper.
clarity,_quality,_novelty_and_reproducibility: The paper is clearly written and coherent to read. The reweighting objective which is meta-learned is not novel, however the authors use it for generation task which is novel. 
summary_of_the_review: Overall, the paper is well-written, has satisfactory novelty and strong results. However, I am leaning towards weak reject as I feel that the authors did not compare with enough baselines where augmentation is used. Given the strong results, I will be willing to increase my scores if the authors can : (i) provide some motivation on why these 3 augmentation techniques were chosen to compare to or add some more baselines to Table 1.  (ii) Some quantitative measurement of the mitigation of label noise on the meta-objective.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper is exploring the few-shot capabilities for pretrained language models, by proposing a two-step method called FewGen: first to generate novel training samples from few-shot samples to augment the original set; and second to perform classification tasks by fine-tuning a classification-based language model on these generated samples. Additionally, the generation of novel samples is guided by a meta-objective which uses a weighting mechanism to choose the most important tokens w.r.t a label and produce more discriminative text samples. The model achieves better results on the GLUE benchmark, which consists of natural language understanding tasks, compared to existing few-shot methods, both with and without data augmentation steps.
strength_and_weaknesses: Strengths

+The idea of combining an autoregressive language model for data augmentation and another classification language model that performs the task, is simple and novel to some extent.

+All used modules and components are reasonable and sound, including the pre-trained models in the framework and the considered datasets/tasks. There is also some additional analysis and ablations included.

+The empirical study shows that the proposed method FewGen outperforms other SOTA methods, with and without data augmentation on the GLUE benchmark.

+The paper has a good flow, is well-written and understandable for the most part.

Weaknesses (and questions):

-There is a lack of experiments without the data augmentation done by the autoregressive language model. To have a better understanding of how the data generation part helps, the authors should provide a baseline where the model is fine-tuned without augmentation. This will offer a fair comparison to the models trained w/o augmentation in Table 1, and also can highlight the contribution of the classification-only part of the method.

-The paper states that “general idea of meta-learning is to formulate a meta objective to enable automatic learning of hyperparameters”, which is not entirely true, since that can be considered only as a subset of the meta-learning goals. In general, meta-learning is aiming to learn good representations and accrue meta-knowledge by observing batches of few-shot tasks. Therefore, it would be good if the authors can elaborate more how their method achieves this, i.e. to explain thoroughly the learning strategy, which is only briefly mentioned to be the online optimization strategy.

-The paper uses a learnable “w” as a feedforward net with 100-dimension hidden layer, but it is not explained why this is the only considered option. For instance, other options such as self-attention layers, do not seem to be considered. Since this part is important for learning the token weights in the meta-weighted objective function, the structure of this part can be further explored, for instance by providing more ablation analysis.

-The paper is lacking more qualitative analysis and discussion. It would be nice to provide more generated samples for given k-shots and for different tasks, and to discuss how they contribute to a better performance.

-It is not entirely clear how the shots are defined in the downstream tasks. Are they predefined in the scope of the datasets? This part can be more elaborated.

clarity,_quality,_novelty_and_reproducibility: Clarity and Quality: The paper is written in a clear and comprehensive manner for the most part

Novelty: The method presented in this paper is mostly combining existing techniques in a novel manner. For example, they combine techniques from Meta-Weight-Net (which first proposed to learn weights as hyperparameters of the objective) and data augmentation to perform more accurate text classification.

Reproducibility: The paper offers sufficient information to reproduce the method, and also the code is available in the supplementary material.

summary_of_the_review: This paper is interesting and proposes to smartly combine existing techniques from data augmentation and meta-learning. However, meta-learning comes a bit of a sudden and is not clearly motivated, nor the learning strategy is thoroughly explained. The authors should try to define the link between the proposed techniques, namely, data augmentation through text generation and meta-learning the weights, to have a more consistent storyline. Also, it would also strengthen the paper if the authors consider the points and concerns raised in the “weakness” part.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper proposes a technique for improving few-shot fine-tuning performance by using the few-shot examples to induce a separate language model to generate additional training samples. The auxiliary autoregressive LM is fine-tuned on the few-shot examples using prefix tuning and a novel loss and optimization procedure that encourages producing text that is consistent with a label. While the proposed method demonstrated improved performance over prior work, reviewers had various concerns, including the complexity of the method (which is especially concerning in few-shot settings where limited data is available for system tuning), the motivation of bi-level optimization, and insufficient baselines. The consensus was therefore to reject this paper.
justification_for_why_not_higher_score: See metareview for concerns.
Decision: Reject