Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: In this paper, the authors propose a novel differentially private approach to generate both continuous as well as discrete valued synthetic data. The authors utilize a one shot approach to providing the privacy, by first generating privatized embedding of the sensitive dataset. The privatized embeddings are then iteratively compared against the synthetic samples generated by the generator module using the characteristic function distance. The approach is similar to and can be considered a generalization of another new approach DP-MERF and seems to compare favorably in empirical experiments against DP-MERF and DP-GAN (two popular alternative approaches). 
main_review: The main strengths of the paper in my opinion are: 
1) The approach is quite well motivated and the paper is well written. 
2) The mathematical formulation is easy to follow and the approach is indeed quite novel 
3) At least in the few empirical evaluations, this approach compares favorably against the other approaches compared against. 

It seems to me, the main weakness of the paper is the lack of thorough benchmarking. For tabular data, it would be nice to see how the approach against PATE-GAN and on more than one dataset. For image data, it would be nice to see the benchmarking done on at least one somewhat more complex dataset (maybe CIFAR-10). 

One minor comment is that the authors should consider moving algorithm 1 to the main paper. 
summary_of_the_review: The paper presents a novel and well motivated approach to privacy preserving generative modeling. The approach has nice theoretical properties and compares well against other approaches on the few benchmark scenarios that the authors have looked at.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: Paper proposes a method for synthetic data generation called PEARL.
main_review: PEARL, a method to generate synthetic data with privacy preservation is proposed. Please see my detailed comments below:

1. I am a bit confused by the auxiliary information release, is the total privacy budget split for this information and the training, or does information is "free"?

2. I am also confused by the sensitivity calculation, especially the factor k, can authors explain where does that come from? Isee they have used it from l2 norm, but please explain how is that equal to k.

3. I see no visible difference when epsilon is increased from 1-10-infinity, does PEARL's performance saturates at a privacy budget?

4. It will be nice to compare PEARL with other generative methods such as PATE-GAN, CTGAN for tabular data etc.

summary_of_the_review: Paper proposes a new method (PEARL) to generate synthetic data based on CFs. It is a nice approach that seems to work.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper studied an important topic in the field of data synthesis: how to train a private deep generative model without reusing the original data. In this paper, the authors proposed a new framework that uses deep generative models to synthesize data in different private ways. Unlike popular gradient cleaning methods, the framework proposed in this paper does not incur additional privacy costs or model constraints. In addition, in order to avoid the problem of reduced privacy guarantee as training iterations increase, this paper used feature functions and adversarial re-weighting objectives to solve the above problems. Both theory and extensive case studies verify the performance of the proposed framework. 
main_review: The strengths and weaknesses of this paper are summarized as follows:
Strengths:
+ The problem studied in this paper is important and needs to be solved in data synthesis
+ Good writing
+ Sufficient theoretical analysis

Weaknesses:
- Need to include more related work that is highly important
- Need more justifications about the novelty claims 
- Need to add more benchmark datasets
- Need to add more advanced baselines
- Lack of complexity analysis

Comments:
1. The following important references are missing:

[1] Zhang Z, Wang T, Li N, et al. Privsyn: Differentially private data synthesis[C]//30th {USENIX} Security Symposium ({USENIX} Security 21). 2021.

[2] He, Xi, et al. "DPT: differentially private trajectory synthesis using hierarchical reference systems." Proceedings of the VLDB Endowment 8.11 (2015): 1154-1165.

[3] Cai, Kuntai, et al. "Data synthesis via differentially private Markov random fields." Proceedings of the VLDB Endowment 14.11 (2021): 2190-2202.

2. How to implement privacy-preserving optimization in this paper is vague. It would be better if the authors could add more details, especially how to perform one-shot sampling. In addition, the authors need to explain how the one-shot sampling strategy guarantees privacy.
3. It would be better if the authors could add 1-2 more public datasets to verify the performance of the proposed framework. For example, the Colorado dataset is the census dataset of Colorado State in 1940, which is used in the final round of the NIST challenge.
4. It would be better if the authors could add 1-2 more baseline schemes (e.g., PrivBayes, PGM, and PrivSyn) to verify the performance of the proposed framework.
5. Like [1], it gives a detailed analysis of the computational complexity of the algorithm. Therefore, it would be better if the authors could give an analysis of the computational complexity of the proposed algorithm.
6. In [1], the authors evaluated the statistical performance of the synthesized datasets on three data analysis tasks (i.e., Marginal Release, Range Query, and Classification). Therefore, it would be better if the authors could add more tasks to verify the proposed algorithm.
7. The editorial quality of this paper is not always satisfactory. It contains quite a lot of inconsistent/non-precise descriptions, as also reflected in the above comments.

summary_of_the_review: The paper tackles a very interesting problem, and the many technical considerations and the experiments on various datasets and comparisons with existing data synthesis techniques are commendable. Some issues (unclear privacy-preserving optimization method, need to add baselines and benchmark datasets) still prevent me from recommending complete acceptance. More clarifications are necessary, and by adding some more "realistic" experiments, I believe the paper could be turned into a significant submission of ICLR. I recommend a "5", but my score can be easily increased to 6 by addressing the many clarifications expressed in my review. Further experiments and the concrete use-case would further improve my score.
review_title: null

Meta-review: The paper presents a new framework of synthesizing differential private data using deep generative models. Reviewers liked the significance of the problem. They raised some concerns which was appropriately addressed in the rebuttal.  We hope the authors will take feedback into account and prepare a stronger camera ready version.
Decision: Accept (Poster)