From: NeurIPS.cc/2022/Workshop/SyntheticData4ML/Paper3/Area_Chair_Ytcf
recommendation: Accept
review: The reviewers agree that the paper proposes an interesting idea and is well-written, hence I recommend to accept this paper. If accepted, I would urge the authors to address the two concerns of reviewer 91pU in the camera-ready version.

From: NeurIPS.cc/2022/Workshop/SyntheticData4ML/Paper3/Reviewer_p4X5
title: Few-shot data augmentation for dialogue understanding
review: In this work, the authors present a few-shot data augmentation framework for dialogue understanding by prompting large pre-trained language models. They further introduce a weakly supervised labeler to filter prospective data points. Models fine-tuned on their augmented data approach or surpass existing state-of-the-art performance on DAILYDIALOG and FACEBOOK MULTILINGUAL TASK-ORIENTED
DIALOGUE datasets.
rating: 7: Good paper, accept
confidence: 4: The reviewer is confident but not absolutely certain that the evaluation is correct

From: NeurIPS.cc/2022/Workshop/SyntheticData4ML/Paper3/Reviewer_91pU
title: Interesting problem to tackle, but doubtful about the motivation of the proposed approach
review: Summary
This paper proposes a filtering technique to select the best quality synthetic data generated by prompts for the dialogue task. They evaluate various augmentation methods on the classification tasks in DAILYDIALOG and the intent detection task of FBTOD.
What the paper addresses an existing problem in dialogue augmentation, however whether it is the correct approach or not is entirely clear to me (see question 1).

##########################################################################

Positive points about this paper

Addressing the problem of out-of-distribution data generation is an interesting idea. Augmentation is a valid approach to address this lack of data in the dialogue task. 
The paper covers the literature well and the experiments showcase the usability of the approach. It is well-written and experiments are clearly defined for the most part. 

##########################################################################

Concerns and questions:

- My main concern for this paper is as follows: The authors say that prompted augmentations are less potent because they lack task-knowledge, which can be distilled from ground-truth (“gold”) samples by training an auxiliary model. They then use that model to filter out inconsistent generated utterances. The filtering model would work with the task knowledge ONLY if there is already relevant and in-distribution data generated by the LM. It doesn't necessarily address the problem of lack of task-knowledge in the augmentation process. Isn't that the main problem this paper is trying to address?

- What is the impact of the prompt you used on the results? How much effort went into engineering the prompt? Did you try different variations or did you not "finetune" the prompt?





rating: 5: Marginally below acceptance threshold
confidence: 4: The reviewer is confident but not absolutely certain that the evaluation is correct

From: NeurIPS.cc/2022/Workshop/SyntheticData4ML/Paper3/Reviewer_JMCz
title: Interesting approach that demonstrates promise
review: Interesting idea to use a classifier post inference in the feedback loop. I found it to be creative, and apparently easy to use as a plug-in method. The consistent improvement across accuracy and F1 scores for the Spanish dataset is also attractive.
rating: 8: Top 50% of accepted papers, clear accept
confidence: 3: The reviewer is fairly confident that the evaluation is correct