Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper extends the PATE approach to the text generation problem. The authors introduce additional steps to help boosting the performance of PATE in the text generation setting as the oiriginal approach does not bode well with the large output space of vocabulary. The paper further studies phrase-level privacy beyond the regularly studied sample-level privacy.
strengths_and_weaknesses: The paper is well-written and easy to follow. Unfortunately my main concern is on the novelty of the paper. The approach is heavily based on the PATE algorithm with few tricks to work better for the text generation task. It utilizes a pre-trained LM to generate pseudo completions and reduces the output space by filtering the tail distribution without a privacy requirement and finally the privacy loss is reduced by acquiring the teacher supervision only when the student is not good at a certain prediction. The latter idea has also appeared in the more recent PATE paper. While I believe these extensions are valuable in improving the performance of PATE in this scenario, I do not think they provide sufficient novelty for this venue.I have one critical comment about the users' secret phrases section. The authors took the route of group privacy for this scenario, which I do not think might bethe effective way with DP. DP-SGD algorithm can easily be adapted to have "user-level privacy" by batching users instead of samples. I find it an unfair comparison in the sense that the authors have not employed this approach but took the naive way of applying group privacy on user-level.
questions: See strengths and weaknesses.
limitations: NA.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper proposes an extension of PATE, a private learning algorithm, to text generation tasks. The extensions are simple yet effective: they generate pseudo inputs and reduce the sequence generation problem to next word predictions. They also propose a strategy to dynamically filter out candidates to reduce the large output space in the text decoder. Experiments in the sentence completion task show that the proposed model is effective in protecting samples and sensitive phrases.
strengths_and_weaknesses: Strengths
* The proposed extension is very simple yet intuitive and effective for differentially private text generation.

Weaknesses
* The paper could have been more convincing if the model is tested on multiple text generation tasks such as dialog response generation (generate a response given previous utterances) where privacy is more crucial.
questions: None.
limitations: As the work focuses on privacy, I think it would be nice to have a specific section on limitations and societal impact. This is currently absent in the main paper.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: In this paper, authors propose a novel framework, SeqPATE, an extension of PATE on text generation, as a Differentially private (DP) learning algorithm for text generations.
SeqPATE aims to protect the privacy of both training samples and sensitive phrases in samples,
and  employs a teacher-student framework.
Additionally, authors propose several strategies for SeqPATE to handle text generations with a sequence of classifications over large spaces.
strengths_and_weaknesses: Strengths

+Privacy protections is important for text generation models and other tasks

+Motivation and problem setting are clear

+Previous work survey is enough


Weaknesses

-Several claims have not been adequately verified. For example, "the effectiveness of SeqPATE in protecting both samples and sensitive phrases", "training corpora with a moderate privacy cost".

-No qualitative and error analysis

-Runtime analysis is lacked. 
questions: *How about applying a simple approach such as a word blacklist to a benchmark?

*Can you explain the rationale for the superiority of this framework compared to the case where teacher models and student model are studied with $D^{pub}$ and $D^{pri}$, respectively?

*Show some examples or qualitative evaluation of how SeqPATE achieves utility with strong privacy protections on training samples.
limitations: Without qualitative analysis, 
it is a quantitative comparison of similar models and does not support the author's claim.
Only the usual text generation model metrics (i.e., PPL and Bleu)are used.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper extends PATE into the field of text generation.  To do so, the following technical challenges must be properly addressed: 1. in addition to protecting individual words, we need to protect phrases too. 2. Compared to other task, the output space is huge for text generation. 3. We need to control the privacy loss.  This paper has done a solid work to address these challenges.
strengths_and_weaknesses: This paper is well written.  This work is original.  It is based on the theory of differentially private (DP), so its potential and quality are pretty high. The important difficult points are well explained.
But as a reader, I think one area can be improved: the connection between theory of DP and the proposed SeqPATE method can be explained more explicitly.  Doing so can greatly decrease the barriers to new researchers who are interested in this area.
questions: This is a solid work.   But making it easier to follow may greatly increase the influence of this work.
limitations: Privacy is an area with great social impact.  This paper focuses solely on the technical aspect of privacy.  But it’s too early to give an assessment on its social impact.  So in my humble opinion, it is acceptable that its social impact are absent in the paper.
review_title: null

Meta-review: metareview: The paper studies PATE framework for text generation models and proposes algorithm based on KD to handle large output space.  Reviewers think that proposed methods should generate interest among the NeurIPS audience. We encourage the authors to incorporate comments of the reviewers to improve the paper.
confidence: Less certain
Decision: Accept