Differentially Private Conditional Text Generation For Synthetic Data Production
Authors: Pranav Putta
Ander Steele
Joseph W Ferrara
Venue: Submitted to ICLR 2023 (2023)
Keywords: differential privacy
conditional text generation
NLP

Abstract: Companies have faced increasing pressure in recent years to anonymize user collected data when sharing internally or to third parties. Text data in particular contains copious amounts of personally identifiable information that has proven to be difficult to de-identify while remain useful for the party of interest. Previous works have suggested that synthetic text generation could provide a promising avenue to curate high performant and private datasets. In this paper, we introduce an approach to synthesize high utility text classification datasets by performing conditional generation through a large language model, distilGPT2, while providing measurable guarantees via differential privacy. We show that naive approaches suffer heavily from utility loss by entangling task-relevant factors in the transformer embedding space, making controlled generation more difficult. We analyze how incorporating a secondary learning objective can improve the performance of the generative model, improving utility of the generated data.

Under review as a conference paper at ICLR 2023
DIFFERENTIALLY PRIVATE CONDITIONAL TEXT GENERATION FOR SYNTHETIC DATA
Anonymous authors Paper under double-blind review
ABSTRACT
Companies have faced increasing pressure in recent years to anonymize user col- lected data when sharing internally or third parties. Text data in particular contains copious amounts of personally identifiable information that has proven to be difficult to de-identify while remain useful for the party of interest. Previous works have suggested that synthetic text generation could provide promising av- enue to curate high performant and private datasets. In this paper, we introduce an approach to synthesize high utility text classification datasets by performing con- ditional generation through a large language model, distilGPT2, while providing measurable guarantees via differential privacy. We show that naive approaches suffer heavily from utility loss by entangling task-relevant factors in the trans- former embedding space, making controlled generation more difficult. We analyze how incorporating a secondary learning objective can improve the performance the generative model, improving utility of the generated data.
INTRODUCTION
In recent years, language models have seen dramatic improvements in performance over NLP tasks. In large part, this has been due to the rapid accumulation of user generated text on the internet. Companies have been able to aggregate millions of documents available online as well as their user data to train these large language models. However, lawmakers and their constituents have grown wary of data collection and usage practices, urging more stringent regulations.
In 2018, the EU set the General Data Protection Regulation (GDPR) into motion, with the goal to increase transparency about collected information and give users more control over how their data is handled. (Voigt & Bussche, 2017). Consequently, companies are now searching for ways to utilize user data without exploiting user privacy. The GDPR begins with the statement: "The protection of natural persons in relation to the processing of personal data is a fundamental right"; it is imperative that we innovate on methods use data effectively without risking user privacy.
In this paper, we study privatization of unstructured text data. Even with safety measures in mind, there has been massive exploitation of user text data. For example, in 2006, as part of their algo- rithm contest, Netflix released a de-identified dataset of user generated movie reviews. Researchers discovered that surprisingly little information was required to reconstruct the identities of users that contributed to the reviews (Narayanan & Shmatikov 2006). Further studies have shown how other methods, such as authorship and membership inference attacks (Carlini et al. 2020), can be utilized to reconstruct user identities. All this to say, without proper privacy guarantees and careful data analysis, companies risk user data to exploitation.
Dwork (2066) and Abadi et al. 2016) proposed differential privacy (DP) and DP-SGD/DP-Adan respectively, as methods to provide provable and quantifiable guarantees about privacy. Generally, say that randomized algorithm satisfies DP if the output distribution is indistinguisable when run on neighboring datasets. However, current trade-offs between privacy and utility, particularly in synthetic text generation, makes it impractical for companies to create useful data with strong privacy guarantees.
A common approach for anonymization is to de-identify (redact) personally identifiable tokens in text, such as names and addresses. While this may seem like a reasonable approach on paper with SOTA models reporting accuracies of nearly than 97%, the 3% of tokens that are misidentified could

---

Under review as a conference paper at ICLR 2023
be used by adversary to re-identify users. Consequently, this approach isn't strong enough guarantee of privacy. A permissible error from such a model should be lower than 1% (Yogarajan et al. 2020; Al Aziz et al. 2021), something that has not been achieved today for abitrary datasets. Synthetic data is promising because avoids the problem of anonymizing an individual's data by instead producing information about non-existent persons.
Other approaches to anonymize unstructured text data have focused on word or sentence level per- turbations order to reduce vulnerability membership inference and authorship attacks. These approaches often heavily degrade semantic quality of the text and may struggle to provide overall privacy guarantees in the context of language peculiarities, such as with the leakage of PII. Other approaches seek to generate data synthetically, such as Libbi et al. (2021) and [Al Aziz et al. (2021). However, such studies often show large tradeoff between privacy and utility or make differentially private guarantees with potentially unreasonable epsilon parameter (e.g.
10).
In this paper, we present an approach of generating synthetic text data by performing controllable generation through large language model. We show it is possible to synthesize text classification datasets with rigorous privacy guarantees. We hope this method will enable companies to share data and train high utility models without putting their users' data at risk. Our contributions are as follows:
1. We present findings on problems that arise when performing conditional finetuning of large language models with DP-Adam. Particulary, we find that it becomes difficult to condi- tionally prompt the model towards a desired class and generate synthetic data that mimics desired attributes of the original. We propose using task-relevant loss via secondary learning objective to solve this issue.
2. We generate synthetic versions of the SST-2 and AG News datasets by performing condi- tional text generation over langauge model. We incorporate a combination of generation techniques: attribute conditioning and gradient based approach (Dathathri et al. 2019) to further steer generation. We show minimal loss in utility of our synthetic datasets 3))% with strong privacy guarantees (e 3).
Code recreate results available here: (redacted for review)
BACKGROUND
2.1 LANGUAGE MODELING
Given sequence of tokens X To. ... Zn language models (LMs) are trained to compute the
unconditional probability of the sequence p(X). This probability can be rewritten in terms product of conditional probabilities by recursively applying the chain-rule (Bengio et al. 2003) as:
x(X) = II p( ..., x;-1)
This allows modeling the language via next-word prediction. We use the transformer architecture Vaswani al.) 2017) to model the distribution of natural language. Generation of new sequence y can be created by scquentially sampling its constituents: pe (yo), pe (y1l30), ..., pe (9m (/
2.2 CONDITIONAL TEXT GENERATION
Conditional generation of text attempts to steer the output of LM given a desired condition or control variable. Keskar al. (2019) introduced method accomplish this goal by performing training a LM over a dataset, such that the desired condition prepended to the text body: "BOS [condition] SEP text" (BOS and SEP are special tokens indiciate the beginning of the sentence and to separate label from the text body, respectively).
On the other hand, plug and play controllable language generation (PPLM) (Dathathri et al. 2019) combines an attribute model (such as discriminator) with LM to manipulate its output and per- form controllable text generation. Given attribute and generated text J, let the output of the
2

---

Under review as a conference paper at ICLR 2023
discriminator model represent p(a).)). In order to control generation, shift the latent hidden state of the language model at step i. h2 by Åh; in the direction of the sum of two gradients: (1) towards smaller cross entropy loss in the attribute model p(a (x) for the desired attribute and (2) toward higher log likelihood of the language modeling head p(x:) to preserve the generation quality and fluency.
In this paper, we use a combination of the two approaches in order to generate high-quality data. We first fine-tune large language model over the desired dataset with conditional prompting similar to Keskar et al. (2019) and then use the gradient-based approach as described by Dathathri et al.) (2019) to steer generation with high likelihood towards the desired attribute. With this process, we can generate labeled data for our synthetic dataset.
2.3 DIFFERENTIAL PRIVACY
Differential Privacy (DP) formal definition of privacy which offers strong assurances against various re-identification and re-construction attacks (Dwork] 2006 Dwork & Roth 2013). In recent years, DP has attracted significant attention due to its mathematically sound and provable privacy guarantecs. Morcover, it has unique properties such as robustness to auxillary information and post- processing, composability to enable modular design, and group privacy. (Dwork & Roth, 2013) Abadi et al. 2016))
2006 A randomized function M provides (c, differential privacy if for all adjacent datasets X,X' X and all Y CV2
Definition 1. (Differential Privacy Dwork
Pr(M(X) E eY exp (c) Pr[M(X') EY
(2)
This is standard definition of DP, which implies that the outputs of DP model/algorithm for neighboring datasets are indistinguishable, bounded by the privacy parameter e. e is a non-negative number which represents the privacy budget. Smaller e values more rigorously enforce privacy, but may have the effect of decreasing data utility. DP also allows for tracking privacy loss throughout the execution of program by computing its leakage parameters. In this paper, we use Renyi Differential Privacy for accounting privacy budget (Mironov] 2017).
Composability and robustness post-processing are important properties of DP that are necessary for the guarantees in our paper. Composability allows for reasoning about overall privacy loss from the composition of multiple DP algorithms releasing multiple statisties about particular dataset. Robustness post-processing implies that if some mechanism M satisfies e-differential privacy, then for any deterministic or randomized function F, so does F(M). This allows us to make e-DP guarantees about the generated text from our e-DP trained language model.
Definition 2. Differentially Private Stochastic Gradient Descent (DP-SGD) modifies the update step during backpropagation by (1) clipping the gradient for each example in the mini-batch maximal norm C and (2) adding Gaussian noise with standard deviation proportional to C to the mean of the clipped gradients.
(t+1)) w() In Σ clipc(VL,(wx)) N(0, REB,
(3)
Where clipc v min(1, TT Intuitively, the DP-SGD mechanism preserves privacy by miti- gating the impact of out-of-distribution samples the model, and is used during fine-tuning of our language models. DP-Adam is the differentially private version of the Adam optimizer (Kingma & Ba 2014), using the same gradient privitization as outlined DP-SGD.
RELATED WORKS
Current methods text privitization fall into three general categories: word/sentence level pertur- bations, private text embeddings, and synthetically generated text. Here, we discuss each method.
Word/Sentence Level Perturbations: Many works have discussed anonymizing text by perturbing word or sentence level embeddings to satisfy e-differential privacy. This set of approaches change individual words in document, often following a variant of metric based DP (Alvim et al. 2018)
3

---

Under review as a conference paper at ICLR 2023
which has shown to be more utilitarian perspective of privacy in the context of NLP. However, as discussed by MMttem et al. (2022). these perturbations struggle to provide overall privacy guarantees in the context of language peculiarities and leakage of other personally identifiable information (PIT) that allow for re-identification. They also suffer from utility losses since grammatical and syntactic structure are degraded. Other methods suggested by Weggenmann & Kerschbaum (2018) and Bo et al. (2019) investigate differentially private mechanisms via latent space perturbations and adversarial training, respectively, to reduce the impact of authorship inference attacks. However, these methods, again, do not address the issue of PII leakage and suffer from significant uility losses.
Private Text Embeddings: Other methods have investigated releasing private text embeddings instead of the original text content. Recent work such as Lyu et al. (2020) and [u et al. (2021) propose randomization mechanisms that can transform text embedding vectors into one that satishes metric space differential privacy guarantees. This method has shown promise in providing formal guarantees while also retaining high utility. However, this process does not leave human readable text, which is desired property for companies performing internal data sharing; thus, we examine our approach independent of this body of work.
Synthetic Text: Other methods, particularly in the medical domain, have attempted to address the issue of privacy via synthetic text generation. Synthetic data addresses the problems of de- identification by simply not describing real people, and thus retaining plausible deniability over the data produced. Recent methods like Libbi et al. (2021) and AI Aziz et al. (202T) have proposed text generation approaches: This paper goes further, investigating the impact of large range of parameter selection in conditional text generation and most importantly, demonstrating high utility even with strong privacy parameters (e.g. = 3), something previous works have not done.
4 DATASETS AND PREPROCESSING
In this paper, we generate artificial datasets for text classification. We choose this task because it allows us best compare utility and privacy in one dataset. We experiment over two datasets. Each dataset is split 80:20 for train and test. We represent datasets as D = { y:)
4.1
SST-2
The SST-2 corpus consists of 11,855 movie review samples, each labeled with positive orn egative sentiment by human annotators. This dataset was perfectly balanced with each class having equal representation (Socher et al. 2013).
4.2
AG NEWS
The AG News corpus is topic classification task. This dataset consists of over 120,000 samples, each labeled under topic from: Sports, World. Business. Sci/Tech. This dataset was perfectly balanced with each topic having equal representation (Zhang et al. 2015).
EXPERIMENTS
This paper improves on existing methods for generating high-utility synthetic text data with differ- ential privacy guarantees. [Bommasani et al. (2019) argued that for successful private synthetic text data, we must have formal guarnatees of privacy and have distributional similarity to the original dataset. We achieve this by conditionally finetuning LM (distilGPT2) over the original text data, the intuition being that we can reconstruct a similar distribution via generation. Since the model is learned privately, the post-processing theorem (Dwork 2006) allows us make the same guaran- tees about the generated samples. We show that with this approach, we are able to construct private, synthetic data that retains high utility. We hope that this will enable companies to utilize synthetic data, reducing reliance on private user information.
All experiments were
NVIDIA V100 GPU instance.
4

---

Under review as a conference paper at ICLR 2023
5.1 FINE-TUNING
The baseline language model that we use for training is a pretrained distilgpt2 from HuggingFace Sanh et al. 2019). We use this model over the larger versions to provide faster iteration of training under different configurations.
We fine-tune the language model G to the task of synthesizing labeled sentences to obtain the fine- tuned language model Ituned- Here, G is specifically fine-tuned to the linguistic domain of Dtrain (that is, the sentences, vocabulary, style, etc.), as well as the particular classes in Derain. The language modeling head, a feed forward network attached to the transformer architecture, is used to model the distribution of the next from an input sequence. During generation, we sample from this head. Generally speaking, with conditioned attribute a being the class label.
would like to use Gtuned to generate sentence set of any length
We fine-tune by training it over the data from Dtrain = {(1,, {=) We generate training samples for conditional finetuning by prepending the label with the text body so that we end up with: U BOS yi SEP zg. We fine-tune this model under different privacy settings, specified by the epsilon parameter. When training with DP, the Adam optimizer is substituted with the DP-Adam optimizer implemented from the tranffrrranssormmrs library provided by Li et al. 2021). We also use the ghost-clipping mechanism outlined by Li et al. (2021) which introduces a memory efficient method to perform per-example gradient clipping. Renyi differential privacy (Mironov 2017) was used to account privacy budget during training.
5.1.1 BASELINE METHOD 1: CONDITIONAL FINE-TUNING WITH FILTER
In our first approach, we (1) perform full fine-tuning of G with the training procedure described above to produce Gruned- (2) We independently train a discriminator to model p(a[z). the probability of generated sample, x, to belong to the class a. In our work, we model the discriminator by fine-tuning language model for classification over the dataset.
(3) We conditionally generate ne samples for each class from G and filter out any samples that do not meet a desired threshold score from the discriminator (e.g. only include the sample if p(a (x) >> 0.5). Specifically, generation was done by performing nucleus sampling (Holtzman et al. 2019) over the output distribution of Guned- The described approach is similar to several methods used in data augmentation (Anaby- et al.,
2019 Bayer et al., 2022 Queiroz Abonizio & Barbon Junior 2020).
This approach worked well for generating artificial datasets for SST-2 and AG News in the non- private setting. We synthesized datasets for each by generating the same number of samples for each class as the original. Generation was done by simply prompting the model with "BOS class SEP". In the private setting, we replaced the Adam optimizer with DP-Adam and tracked the total privacy budget with the RDP accountant. As we improved the privacy guarantee with smaller epsilon parameters (e.g. c 8), the quality of conditional generation quickly degraded. While the private LM generated text that appropriately mimicked the linguistic domain of the training data, condi- tional prompting did not produce consistent results; prompting the model with attribute would infrequently meet the threshold requirement from p(alar).
We also analyzed samples qualitatively and found the same results. For example, the non-private Guned generally produced samples that fit the class it was prompted: (c.g. "BOS positive SEP" might yield "a sensitive and heartwarming story of aging man...".. However, the same approach with the private Gumed produced samples that very inconsistently fit the prompted attribute (e.g. "BOS positive SEP" might yield "an emotional slap in the face, and..."). See Appendix B for more examples. Without having high confidence in our model being able to generate text conditionally for a desired class, the labels in the synthesized dataset may be meaningless. This would severely degrade the utility of the artificial data. This result suggests that a stronger mechanism than just prompting is required steer the model towards high-quality class conditional samples.
5.1.2 BASELINE METHOD CONDITIONAL FINE-TUNING WITH PPLM GENERATION
Iterating from Baseline 1, we attempted to use a similar approach as PPLM DDthathri et al. gradient based steering mechanism, to guide the private Gunndd models towards more high qual-
2019
5

---

Under review as a conference paper at ICLR 2023
Finetune (DP-S62)
Dueek's aeddiiegiana
GPt-2
todeling
Iclentiate
Discrininator (DP-SGD)
PFLN-style Generation
tabeddinge
Bead
6P
DELARAL
ity generation. Similar to Baseline 1, we (I) train Guned, then (2) train a discriminator to estimate the attribute model p(a (x) by training discriminator head over the frozen Guned model. The dis- criminator head is a simple MLP with non- non-linear activations. Lastly, (3) we perform PPLM-base conditional generation (See Section 5.2) generate the synthetic labeled text classification dataset.
The intuition for this approach is that the gradient based generation mechanism will guide Guned into generating samples that align strongly with the desired label. In order to effectively the discriminator to perform gradient updates on the hidden states of Gruned we trained the discriminator over the fine-tuned LM's frozen embeddings. Again, while this approach worked well in the non- private setting, it became infeasible to train the discriminator at strong epsilon settings, At 3, 8 the discriminator was not strong enough to properly contribute to generation. We hypothesized that this issue was indicative that Grnnd was not preserving information about the attribute labels during private fine-tuning, making difficult for the discriminator to learn separation, and simulatenously making it more difficult for the I.M to generate label aligned samples as observed in the previous section. We investigated this hypothesis by visualizing the embedding space Guned at different epsilon settings and estimating the mutual information between the transformer embedding space and class labels by training a Random Forest classifier (See Figure 1). We hypothesize that in order to strongly reconstruct distributional properties from the original dataset, the generative model should produce embeddings that are separable with respect to those task-relevant factors.
5.1.3 OUR METHOD: MULTITASK CONDITIONAL FINE-TUNING WITH PPLM GENERATION
Pinetune Nodel
Zepet Ietaret
style Coneration
decadlse
Aiceling
allee
spsttetie
cliee Newn harrys
(tertics)
Siscstadassee
In order to address this issue we introduce a secondary learning objective and perform multitask learning during fine-tuning. In Baselines 1 and 2, the transformer only attached linear lan- guage modeling head that models the probability distribution of the next word. In our approach, we simultaneously train discriminator head, as shown in the diagram above. The discriminator head is, like Baseline 2, simple MLP head. We now perform two gradient updates at every step one to update the language modeling head and the other update the discriminator head. We add the appropriate amount of noise to the gradients to maintain c-DP guarantees and track privacy budget throughout training with RDP [Mironov] 2017))
Since still want to retain conditional prompting for the model, we want the language model to be able to see the conditional prompt, i.e. "BOS positive SEP text", which includes the prepended label so that the model is able to understand prompting. Meanwhile, the discriminator head should be able to learn to model p(a|z) for a label, a, and generated sample x without seeing the label in the input. So, for the language head, feed the label prompted text data and perform gradient

---

Under review as a conference paper at ICLR 2023
DP Guarantee
Baseline
Ours
6 inf
0,803
0,883
256
0.792
0.873
6 16
0.773
0,869
poethe
0.739
0.865
0.693
0.866
Figure 2: Random Forest Classifier Test Accura-
cies over SST2 Embeddings from Guned The mul- titask approach (ours) shows marginal loss in per- formance high privacy settings.
Figure I: UMAP Projection SST2 Embeddings from Gunodwith 3. Baseline 2 (top). Ours (bottom).
update. Then, for the discriminator head, we replace the label in the input with random token, the intuition being that the discriminator head will pay less attention to the embeddings at that location, and more informative guide during generation.
We also train this discriminator head to classify text at different prefix lengths. For example, if the prefix step was specified to be 2, we would compute the loss given the transformer output for the second token, fourth token, sixth token, and prefix is weighted the least and the last prefix is weighted the most. Lastly, this loss is averaged, and then the gradient update is computed. This loss procedure is ensure the discriminator head is robust enough to provide meaningful classifications at different lengths of sequence to improve its contribution during gradient based generation.
The loss is linearly weighted such that the first
Algorithm DP Multitask Conditional Training
Data:
Greeteinee Daain {0 y number of iterations T, learning rates Thm ]diserim- noise
multiplier a, clipping bound C, initial parameter vectors ((0) ((0)) 0(0) diserim batch size B,
initial moment estimates mo. Ub E", exponential decay rates 31. 32 IR and constant
for 1 E [E B] do
Draw batch b, from D with sampling probability 4-
for (zi, y:) E do
rand + random token from vocabulary
9mm "BOS Yi SEP x.,".
Sdiscrim
"BOS rand SEP xx,"
9m VL(G
(+)
gits tmanst. In
(smm); stm). 9 ddssriim
VL(G
(Sdiscrim): yi)
trasd dianni
9im - gim
(1)
(1)
min(1, C/Ils (0) l12).
Adiserim
Idiscrim
(t)
min (1, C/Ilg ll
end
Iim { Am ( + O00,0
(Eie,,
(e) Se, Jdiserim + N 000,020211
Idiscrim (++1)
transt. f- Adam frans, Im: M4, Pum BI B2,
(1)
(f+1)
AdamUpdatel transt, discrim: Ma, disetiim: 31, (2.7)
transf, discrim
end
Output: Trained Model (fF) ( (0
discrim

---

Under review as a conference paper at ICLR 2023
Ultimately, we find that by training both the discriminator and language modeling head simulta- neously, Gruned is able to conditionally generate even when trained with strong privacy guarantees. In Figure 1, we show how this approach impacts the embedding space of models trained at rigor- ous privacy constraints compared to the naive approach via a UMAP projection. We find that the noise injected via differential privacy doesn't prioritize the model implicitly learn particular dis- tributional factors about the original dataset such as separation of class labels, and an explicit loss mechanism can recover this and improve quality of generation.
5.2 GENERATION
Next, we describe in detail the conditional generation procedure to synthesize private version of the dataset. We aim to generate labeled samples text that reconstruct similar distributional properties as the original. In order to guide generation towards particular class, we apply PPLM (Dathathri jet al. 2019) based gradient approach. We utilize the discriminator trained in the previous step to perform gradient updates over the hidden states of the model to steer the generation towards the desired class. The steps for generation of a single sample are as follows:
1. Prompt the model with BOS class SEP and generate the distribution of the next word via the language modeling head.
2. Compute hidden embedding states of the generated text. Pass this embedding through the discriminator, which models p(a[z))
3. We now shift the hidden state, h; by summing two gradients: (1) gradient of the cross entropy loss between the discriminator output and desired class vector. (2) gradient towards the higher log likelihood of the language modeling head which models p(z)). This is done by minimizing the KL divergence between the modified and unmodified language mdoeling head distribution.
4. Compute the new LM head distribution from the updated latent space.
5. Sample from the new language modeling head distribution for the next word by performing nucleus sampling (Holtzman et al. 2019).
6. Repeat steps 1-3 until the termination token the specified maximum length is reached.
We discuss further implications and limitations of this approach in Section
EVALUATION
With the described approach, generate synthetic versions of the SST-2 and AG news dataset. 5 variations are generated with different diffcrential privacy settings: E {256, 16, 8, 3} and a non- replacing the
private version. The only change between the non-private and private versions optimizer from Adam to DP-Adam provided by the private-trassfommers library (Li et al.) [202T). The gradients in the non-private version are still clipped to the maximum gradient norm parameter, C.
6.1 PRIVACY
Differentially private training provides formal guarantees about the generated text consequence of the post-prov prst-processing theorem. However, recent works have shown that the impact of epsilon DP on large language model training is still unclear, and we could observe empirical privacy- preseyypresevvation even at high epsilon levels. To test this, we test the artificial dataset for memorization by comparing the proportion of n-grams (for n E [3...7) the synthesized data to those present in the original dataset. Our findings are consistent with previous studies with language modeling. Empirically, we see even large epsilon settings dramatically decrease memorization in the synthesized data (Pono- mareva et al. 2022).
6.2 UTILITY
We measure the utility of the synthetic dataset by training a classifier over the synthesized data and evaluate the performance on the held-out test dataset. We don't experiment with different classi-
8

---

Under review as a conference paper at ICLR 2023
5ST2 N-gram precision
AG News N-gram precision
w22 syrtheticees- eps-inf
ne.ns-yyttheekc
#g. news synthetic eps-256
0.00
fe_news--ag synthethhhhcccccmm eps-
222-oytheticcepps.
newnew sytt--yythhtii
s22 #2--oythetic.epps- 3ps--
ag. news www.ssytttctcpppsse.
0.06
0.04
0.02
n-gram
n-gram
Figure 3: N-gram ratios of the different syn- thetic datasets trained on SST-2
Figure 4: N N-gram ratios of the different syn- thetic datasets trained on AG News.
fication models since our goal is to strictly evaluate the performance of the synthesized dataset. So, choose to state of the art classifier, DisilBERTToSSqquunCCCaiiifiiiii from the HuggingFace transformers library.
We first train classifier over the original dataset produce baseline accuracies to compare the utility of the synthetic data to. Next, for each dataset variant, c E (inf, 256, 16, 8, 3}, we train a classifier. To measure the performance of the model, compute the accuracy of the model over the held out test set. These results are shown in Table We do not modify any hyperparameters of the classifier for each dataset. The selected parameters can be seen in Appendix A.
Table 1: Classification Accuracies of Our Method (Multitask Model)
Dataset Variant
SST-2
AG News
Original (baseline)
0.941
0.938
Synthetic
0.892
0.913
Synthetic 256
0,883
0,883
Synthetic 16
0.864
0.874
Synthetic = 8
0.829
0.871
Synthetic
0.803
0.867
DISCUSSION
In this paper, we propose a method for generating synthetic text classification datasets with differ- ential privacy guarantees by performing conditional text generation via large language models. We show the difficulties in doing this naively, particularly exploring how strong settings of privacy im- pact the conditional prompting scheme which has performed well in non-DP settings. By utilizing task-relevant second learning objective and gradient based steering of generation towards desired class, we show conditional generation is possible even at strong privacy settings. We believe this method has potential for creating synthetic datasets that will enable companies to share and train on information without putting users' personal information risk.
However, we want to point out some limitations and future directions for this line of work. Firstly, previous studies have shown that training neural network models with DP-SGD can result in in- creased bias Bagdasaryan et al. (2019). In our work, we chose to use perfectly balanced datasets in order to mitigate the problems of unequal representation faimess issues when generating synthetic data, and biases from the original data may be amplified in the new dataset (Kuppam et al. 20199 Zhu et al.; 2020). Future work may investigate how using this method affects fairness among groups represented in dataset.
classes. This could potentially lead to

---

Under review as a conference paper at ICLR 2023
REFERENCES
Martin Abadi, Andy Chu, Ian Goodfellow, H. Brendan McMahan, Dya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. ACM, oct 2016. doi: 10.1145/297679. 2978318. URL https://doi. http:///o....oo.....141/21121111 www.11129967444.... 2978311
Md Momin AI Aziz, Tanbir Ahmed, Tasnia Faequa, Xiaoqian Jiang, Yiyu Yao, and Noman Mo- hammed. Differentially private medical texts generation using generative neural networks, ACM Transactions on Computing for Healthcare (HEALTH), 3(1):1-27, 2021.
Mário S. Alvim, Konstantinos Chatzikokolakis, Catuscia Palamidessi, and Anna Pazii. Metric- based local differentiaa privacy for statistical applications. CoRR, abs/1805.01456, 2018. URL
hhtt:://arkie www.:orggg/bb.ooooeo http/bbb//11000 01456
Ateret Anaby-Tavor, Boaz Carmeli, Esther Goldbraich, Amir Kantor, George Kour, Segev Shlomov, Naama Tepper, and Naama Zwerdling. Not enough data? deep learning to the rescue!, 2019.
URL httttt/////axiii http:///aawvvvv.ogaaa/////s///// /91.. 03118
Eugene Bagdasaryan, Omid Poursaeed, and Vitaly Shmatikov. Differential privacy has disparate impact on model accuracy. Advances in neural information processing systems, 32, 2019.
Markus Bayer, Marc-André Kaufhold, Björ Buchhold, Marcel Keller, Jörg Dallmeyer, and Chris- tian Reuter. Data augmentation in natural language processing: novel text generation approach for long and short text classifiers. International Journal of Machine Learning and Cybernetics, pp. 1-16, 2022.
Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Janvin. A neural probabilistic language model. J. Mach. Learn. Res., 3(null): 133-)115,, mar 2003. ISSN 1532-4435.
Haohan Bo, Steven H. H. Ding, Benjamin C. M. Fung, and Farkhund Iqhal. ER-AE: differentially- private text generation for authorship anonymization. CoRR, abs/1907 00866,,,., 2019. URLhttpp 7 //aaxiv..rg///aaaaiv.
907. 08736
Rishi Bommasani, Steven Wu, and Xanda Schofield. Towards private synthetic text generation. In NeurlPS 2019 Machine Learning with Guarantees Workshop, 2019.
Nicholas Carlini, Florian Tramèr, Eric Wallace, Matthew Jagielski, Ariel Herbert- Voss, Katherine Lee, Adam Roberts, Tom B. Brown, Dawn Song, Ülfar Erlingsson, Alina Oprea, and Colin Raf- fel. Extracting training data from large language models. CoRR, abs/2012.07805,, 2020. URL
https: /arpsi///axiv.... org/ abs 2012. 07805
Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Picro Molino, Jason Yosin- ski, and Rosanne Liu. Plug and play language models: A simple approach to controlled text generation. arXiv preprint 1912:000 2019.
Cynthia Dwork. Differential privacy: A survey of results. In Lecture Notes in Computer Science, pp. 1-19. Springer Berlin Heidelherg, 2006. doi: 111109978-33-54-7222888888 URL https:
//doi.
Cynthia Dwork and Aaron Roth. The algorithmic foundations of differential privacy. Foundations and Trends@ in Theoretical Computer Science, 9(34):2211 407,, 2013. doi: 10.1561/0400000044. URL https: /tttttts:://doi
/ /tw./ /tt////i..../11111111111111//110
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. The curious case of neural text degeneration. arXiv preprint 191.v:1904.09751, 2019, 2019.
Nitish Shirish Keskar, Bryan McCann, Lav R. Varshney, Caiming Xiong. and Richard Socher. CTRL: A conditional transformer language model for controllable generation. CoRR, abs/1909.058523 05059, 2019. URL //tpxvaaavvvv///v///// www.voorrg.oooo htrgg/ab//ae abs 19099 05858
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv: 22222,6900 2014.
10

---

Under review as a conference paper at ICLR 2023
Satya Kuppam, Ryan McKenna, David Pujol, Michael Hay, Ashwin Machanavajjhala, and Gerome Miklau. Fair decision making using
data. CoRR, abs/1905.12744, 2019. URL
org/ / abs .
Xuechen Li, Florian Tramèr, Perey Liang, and Tatsunori Hashimoto. Large language models can be strong differentially private learners. CoRR, abs/2110.056779 2021. URL https://arxiv:
org/ htt//ba/22111 01677.0606679
Claudia Alessandra Libbi, Jan Trienes, Dolf Trieschnigg. and Christin Seifert. Generating syn- thetic training data for supervised de-identification of electronic health records. Future Inter- net, 13(5):136, May 2021.
doi: 00.3390/1130501333 URL ott.//00.3..0000033333.3
E113050136
Lingjuan Lyu, Yitong Li, Xuanli He, and Tong Xiao. Towards differentially private text representa- tions. CoRR, 1et/22006.111000 2020. URL
org/ wttp:/bbbb.....11111111
Justus Mattern, Benjamin Weggenmann, and Florian Kerschbaum. The limits of word level dif- ferential privacy. In Findings of the Association for Computational Linguistics: NAACI. 2022, pp. 867-881, Seattle, United States, July 2022. Association for Computational Linguistics. doi: 10.18653/v hww.:o2/..iiinn--aaaaaaea naacl.65.
URL httpss//aallaattaoooo..ooo//////
Findings naacl. 65
Ilya Mironov. Renyi differential privacy. CoRR, abs/1702.07476, 2017. URL http::/arxiv.
org/aba/1702. 07476
Arvind Narayanan and Vitaly Shmatikov. How break anonymity of the netlix prize dataset. CoRR, abs/cs/0610105. 2006. URL htttt:::/arxlv..or//bb/////
Natalia Ponomareva, Jasmijn Bastings, and Sergei Vassilvitskii. Training text-to-text transform- ers with privacy guarantees. In Findings of the Association for Computational Linguistics; ACL 2022, pp. 2182-2193, Dublin, Ireland, May 2022. Association for Computational Linguis- tics. doi: 10. (0.118653/v1/2222 http:///////22.....nnnnooo.l...o findings-acl.171 UtL a/cc/aacaaacc www.lnnnntoogo...oooooo org/2022. Findings-aci. 171
Hugo Queiroz Abonizio and Sylvio Barbon Junior. Pre-trained data augmentation for text classifi- cation. In Brazilian Conference on Intelligent Systems, pp. 551-565. Springer, 2020.
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. In NeurIPS EMC W orkshop, 2019.
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, sentiment
and Christopher Potts. Recursive deep models for semantic compositionality treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 1631-1642, Seattle, Washington, USA, October 2013, Association for Computa- tional Linguistics. URL https: www. acl http://nnhooooly/b11---- 1170
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. CoRR, abs/1706.03762. 2017.
URL HtRLhttp::/ /ttpht:::://ii....//// org /1w.. 03762)
Paul Voigt and Axel von dem Bussche. The EU General Data Protection Regulation (GDPR): Practical Guide. Springer Publishing Company, Incorporated, 1st edition, 2017. ISBN 3319579584.
Benjamin Weggenmann and Florian Kerschbaum. Syntf: Synthetic and differentially private term text mining, CoRR, hbbs/180500000 2018. URLhttpp
frequency vectors for 809000004000000
Nan Xu, Oluwaseyi Feyisetan, Abhinav Aggarwal, Zekun Xu, and Nathanael Teissier. Density- aware differentially private textual perturbations using truncated gumbel noise. The International
FLAIRS Conference Proceedings, 34(1), April 2021. doi: 133.f/aair..341..22288844 URL v3411. 128463.
https://80ii.rr//1.337777 httts:://do.... http://o..org//..333333

---

Under review as a conference paper at ICLR 2023
Vithya Yogarajan, Bernhard Pfahringer, and Michael Mayo. review of automatic end-to-end de-
identification: Is high accuracy the only metric? Applied Artificial Inteligence, 34(3):251-266 2020.
Xiang Zhang, Junbo Jake Zhao, and Yann LeCun. Character-level convolutional networks for text classification. In NIPS, 2015.
Keyu Zhu, Pascal Van Hentenryck, and Ferdinando Fioretto. Bias and variance of post-processing in differential privacy. CoRR, abs/2010.04327, 2020. BBK/htttess//arri..orr////////// 04327
A HYPERPARAMETERS AND TRAINING RESULTS
Table 2: Hyperparameters for Training LM used for results in Section 7
Method
Value
Clipping Norm C
0.1
Batch Size B
128
Learning rate
5e-4
Learning rate decay
yes
Epochs E
Last N Frozen Transformer Layers
DP Guarantee
(c, Peaml
Table 3: Hyperparameter search range for experiments
Hyperparameter
Value
Clipping Norm C
(..1, 0.5}
Batch Size B
{16, 32, 64, 128]
Learning rate !)
{200, 100,50, 30. 1e - yes, no 5,4,5,6,7.88
Learning rate decay
Epochs E
Last N Frozen Transformer Layers DP Guarantee
2,3,4,5,61
(e, P
Table 4: Test loss for SST-2 LM results in Section 7
DP Guarantee
Loss (Naive)
Loss (Multitask)
inf
3.823
3,936
256
3.807
3.729
16
4.033
3,812
8
4.102
3,883
= 3
4.183
3.907
Overall, we found that the only hyperparameters that had significant impact on the performance of the language model was learning rate and batch size, consistent with other works.
12

---

Under review as a conference paper at ICLR 2023
Table 5: Hyperparameters for classifier used for results in Section 7
Hyperparameter
Value
Batch Size
32
Epochs E
Learning rate 7/
le
Last N Frozen Transformer Layers Learning rate decay
yes
Table 6: Text Samples From SST2 Naive Model (c inf)
Prompt
Text
BOS positive SEP
de force showcase that combines deep emotional connection with everyday events
sensitive and heartwarming story of an aging man who switches bodies to new places and lives to embrace his enviromment.
cinematic poetry and soulful poetry combine together in stunning, horrifying and moving images and emotions of childhood.
BOS negative SEP
frida lacks emotional center and strong emotional center.
has a grisly undercurrent of a film no humor, no suspense, no intrigue, no suspense, no suspense.
predictably soulless and hokey, and not really funny.
B TEXT GENERATION EXAMPLES
When performing generation thorugh the naive model with DP guarantees, we noticed that was often unpredictable if the model would output text according to its conditional prompting. This is undesirable when generating text for a synthetic dataset, where the samples need to be generated for particular class. We that the output is much more consistent in our approach with the multitask model. This is evidence that separating transformer embeddings with respect to task-relevant factors enables more consistent generation towards desired class.
13

---

Under review as a conference paper at ICLR 2023
Table 7: Text Samples From SST2 Models (e = 3)
Prompt
Naive Model Text
Multitask Model Text
BOS positive SEP
there something genuinely in-
the latter gives a shining, serenely sunny but pleasantly funny story with deep sense of humor.
teresting about this movie this is what it does but you are holding more recent movie up. not as great as you could have hoped for but rather, but it leaves the essence of my greatest hour.
an act creative genius that builds both wide-angle of film and deft blend of comedy.
it is gripping action thriller which is a solid work of humour that hits great length and is worth checking out.
think this is one of the most gripping films of all time and makes me the most disappointed guy you have ever seen.
BOS negative SEP
this is the most brilliant film ever, probably because is sopositive, absurd mix of funny tales from a writer
the show's biggest failure is its own unproduced.'s failure to get its audience impression, as well as his incompetence.
the end of the film draws the line between how flawed and depress- ing it is.
the characters are often por- trayed as the protagonists' par- ents who are misunderstandingly disappointed by the lazy and lazy characters.
it is impossible not giggle at the idea that an audience just liked the film.
the ending has nothing to do with the world history.
14


Reviews:
[{"review_id": "Kot15urfk09", "reviewer": "Reviewer_hB6f", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper tackles the problem of sharing text data by utilizing differential privacy methods. They show how a naive approach does not give good text aligned with the class labels but their new proposed architecture does. Experiments showcasing the efficacy of their approaches are shown by using a pretrained GPT-2 model and adapted on to two datasets namely SST-2 and AG news.", "summary_of_the_review": "Overall, a good practical paper which would be interesting for the synthetic data community. In terms of novel ideas, the impact may be modest.", "strength_and_weaknesses": "Pros:\n\n   (A) They are able to generate high-fidelity and high-utility synthetic data as shown on two datasets in the experiments section.\n   (B) The novel multi-task learning approach gives them the additional boost over the naive sequential approach. Also, the UMAP projections shows the clear separation between the positive and negative labels.\n\nCons:\n\n   (i) There is not much novelty in terms of the proposed approach. Existing DP settings and code (RDP) are used for implementing the privacy aspects and similarly the multi-task learning is also relatively straightforward. \n   (ii) There is still substantial gap between original and the synthetic versions (0.94 vs 0.89) and it would have been interesting to address that gap even before attempting the DP guarantees. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "HH1rSvmk22", "reviewer": "Reviewer_EbEe", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a synthetic data generation method for differential privacy guarantee for large language models. More specifically, the author applies the PPLM-based gradient approach with a discriminator. The proposed method is evaluate on SST-2 and AG datasets.", "summary_of_the_review": "The paper targets to solve an important question about differential privacy for language models. However, the paper is not well organized and prepared to demonstrate the effectiveness of the proposed method.", "strength_and_weaknesses": "Strength:\n\nThe paper addresses an important problem of differential privacy for language modeling. The motivation is clearly stated.\n\nWeaknesses:\n\n1. The paper is not well organized. The proposed method is described in the experimental part, which makes the reader confused about the motivation and the target of the paper. The main sampling strategy is based on a prior work called PPLM. However, the PPLM is not well introduced, which makes the method part hard to understand for the reviewer without prior knowledge about this method.\n\n2. Some baselines are not mentioned and compared in the experimental part. For example, [1] also discussed the differential privacy problem for language modeling. \n\n[1] Shi W et al. , Selective Differential Privacy for Language Modeling, NAACL 2022", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "GUtHNEGFpE", "reviewer": "Reviewer_fPMV", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper aims at synthesizing datasets from private data, using differential privacy. The main goal is to generate data for NLP classification tasks, and the approach this paper takes is to do so through conditional generation, as opposed to just training a generative model with DP-SGD and then taking free form samples from it. The paper discusses three main methods, two as baselines and one as the main proposed method: the first baseline is augmenting the private, training data with its class label (for instance if positive class, you pre-append with a positive token, if negative sentiment class, negative token), which means during generation you would basically prompt the model with the class label and have it generate. The second baseline improves on this by adding controllable generation (using PPLM), which uses a discriminator at decoding time to help enforce given attributes, for example sentiment. So apart from the prompt, the PPLM controllable generation would also help enforce the attribute. This setup is two stage, as in first the generator is trained and then the discriminator for PPLM is trained on top of it. Finally, they propose their method, which uses multi-task learning for training both the language generator and the discriminator at the same time, as opposed to consecutively, to help improve separability of the representations learned by the main model, thereby improving its downstream utility on the classification task.", "summary_of_the_review": "I have provided extensive details of my summary under strengths and weaknesses, but to summarize, I find the paper's problem and approach really interesting. However, I find the presentation and the experimentation extremely lacking, and I feel like the paper needs much thorougher setup and ablations in order for the claims to have merit.", "strength_and_weaknesses": "Strengths:\n1.\tThe paper studies a very relevant problem, as synthesizing DP data (especially text) could have many applications, especially on enterprise level.\n2.\tThe proposed method of conditioning on class label and using controllable generation is new and has not been explored before.\n\nWeaknesses:\n1. The paper lacks experiments that would help make its case and show the superiority of the proposed method, and how it is actually better, given the overheads it has. More specifically, these are the questions I have regarding experiments:\na.\tTable 1, which is basically the main downstream evaluation result, seems to only show results for the proposed method, and not baselines 1 and 2. This makes this evaluation not very helpful, as we don’t know how the baselines are doing, and how much improvement we are getting from what trick. To clarify: there should be comparisons with at least one baseline, and ablations against others so we can see how much benefit a) conditioning on label b)PPLM and c) multi-task learning have. Right now we don’t even know if the proposed method does better than vanilla DP-SGD training a transformer-based classifier on the generated text. The only comparison we do get with one of the baselines is in Figure2, where the experimental setup is not even explained, but it seems to be a comparison of embeddings of the methods, using Random Forrest and is not an end-to-end downstream comparison. \nb.\tThe overheads of the proposed method are not at all discussed. By overhead I mean any extra cost, at training or inference. For instance decoding (inference) with PPLM is actually quite expensive (due to the gradient flow and backprop, and also using a discriminator) compared to non-controllable generation. These costs are not at all discussed/explored/measured in the paper\nc.\tThe paper has missed a very relevant related work, submix [1] which tackles the same problem of DP generation of text. I think a comparison with this paper is necessary.\n\n2. The privacy budgeting of Baseline 2 is not at all discussed: given how this is a two stage approach, where outputs from stage 1 + input to stage 1 (the labels) go to stage 2, the privacy accounting is actually non-trivial. A conservative approach would be to compose the budgets used for stage 1 and 2, as we are re-using the labels. However I am sure better accounting could be done. This affects the reported $\\epsilon$ and therefore the privacy utility.\n\n3. Using N-gram counts as a measure of privacy is very unorthodox and uninformative as privacy is not necessarily violated if a unigram is regurgitated. Researchers usually use metrics like recall of Membership Inference Attacks [2-4], exposure metric [5], extraction attack success [6]. \n\nMinor issues:\n1.\tThe figures explaining baseline 2 and the proposed method don’t really have captions or labels.\n2.\t Figures 3 and 4 aren’t referenced anywhere in the text. I assume they relate to section 6.1?\n\n\nRefs: \n[1] Ginart, Antonio, et al. \"Submix: Practical private prediction for large-scale language models.\" arXiv preprint arXiv:2201.00971 (2022).\n\n[2] Shokri, Reza, et al. \"Membership inference attacks against machine learning models.\" 2017 IEEE symposium on security and privacy (SP). IEEE, 2017.\n\n[3]  Mireshghallah, Fatemehsadat, et al. \"Quantifying privacy risks of masked language models using membership inference attacks.\" arXiv preprint arXiv:2203.03929 (2022).\n\n[4] Carlini, Nicholas, et al. \"Membership inference attacks from first principles.\" 2022 IEEE Symposium on Security and Privacy (SP). IEEE, 2022.\n\n\n[5] Carlini, Nicholas, et al. \"The secret sharer: Evaluating and testing unintended memorization in neural networks.\" 28th USENIX Security Symposium (USENIX Security 19). 2019.\n\n[6] Carlini, Nicholas, et al. \"Extracting training data from large language models.\" 30th USENIX Security Symposium (USENIX Security 21). 2021.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "E7ltN5724p", "reviewer": "Reviewer_Hw7g", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper propose a DP-SGD based learning algorithm to achieve synthetic data generation. The proposed method achieve DP to protect the privacy of the training data.", "summary_of_the_review": "This paper does not meet the borderline of ICLR.", "strength_and_weaknesses": "1. This paper simply combines existing methods (DP-SGD and GPT-2). The novel and the technical contribution to the community is limited. \n\n2. The model performance (utility) is far from real use. If the \\epsilon reaches a meaningful range (<10), the performance drops a lot compared with the baseline (in Table1).\n\n3. The related work section misses some important papers or fails to cite some papers..\n3.1. GPT-2 + DP in text generation.  SeqPATE: Differentially Private Text Generation via Knowledge Distillation. I admit this paper was only accepted by NIPS'22 just now. The authors are not required to compare this paper as a baseline. However, it's necessary to discuss the difference between your paper and this paper.\n\n3.2. GPT-2 + DP-SGD: Xuechen Li, Florian Tramèr, Percy Liang, and Tatsunori Hashimoto. Large language models can be strong differentially private learners. In ICLR, 2022. This paper was published at ICLR'22 instead of arxiv only.\n\n3.3.GPT-2 + DP-SGD:  Da Yu, Saurabh Naik, Arturs Backurs, Sivakanth Gopi, Huseyin A Inan, Gautam Kamath, Janardhan Kulkarni, Yin Tat Lee, Andre Manoel, Lukas Wutschitz, et al. Differentially private fine-tuning of language models. In ICLR, 2022\n\n4. Some competitive DP-based baselines are missing . The technique proposed in [Xuechen Li' ICLR'22] and [Da Yu' ICLR'22] can be easily used in this method. \n\n5. The organization of this paper should be improved.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject