DP-LFlow: Differentially Private Latent Flow for Scalable Sensitive Image Generation
Authors: Dihong Jiang
Sun Sun
Venue: Accepted by TMLR ()

Abstract: Privacy concerns grow with the success of modern deep learning models, especially when the training set contains sensitive data. Differentially private generative model (DPGM) can serve as a solution to circumvent such concerns by generating data that are distributionally similar to the original data yet with differential privacy (DP) guarantees. While GAN has attracted major attention, existing DPGMs based on flow generative models are limited and only developed on low-dimensional tabular datasets. The capability of exact density estimation makes the flow model exceptional when density estimation is of interest. In this work, we will first show that it is challenging (or even infeasible) to train a DP-flow via DP-SGD, i.e. the workhorse algorithm for private deep learning, on high-dimensional image sets with acceptable utility, and then we give an effective solution by reducing the generation from the pixel space to a lower dimensional latent space. We show the effectiveness and scalability of the proposed method via extensive experiments, where the proposed method achieves a significantly better privacy-utility trade-off compared to existing alternatives. Notably, our method is the first DPGM to scale to high-resolution image sets (up to 256 × 256). Our code is available at https://github.com/dihjiang/DP-LFlow.

Published in Transactions on Machine Learning Research (10/2023)
DP-LFlow: Differentially Private Latent Flow for Scalable Sensitive Image Generation
Dihong Jiang
dihong-jionge banong.-iong@uuateelooo
Deportment of Computer Science University of Waterloo
Sun Sun
swn. www.summ.Orr--ccrrrgg...
National Research Council Canada University of Waterloo
Reviewed on OpenReview: openrevvee............ net/ por//f id= GBcneTllyy 119M
Abstract
Privacy concerns grow with the success of modern deep learning models, especially when the training set contains sensitive data. Differentially private generative model (DPGM) can serve as a solution to circumvent such concerns by generating data that are distributionally similar to the original data yet with differential privacy (DP) guarantees. While GAN has attracted major attention, existing DPGMs basod on flow generative models are limited and only developed on low-dinensional tabular datasets. The capability of ezact density estimation makes the flow model exceptional when density estimation is of interest. Tn this work, we will first show that it is challenging (or even infeasible) to train DP-flow via DP- SGD, i.c. the workhorse algorithm for private deep learning, on high-dimensional image sets with acceptable utility, and then we give an elfective solution by reducing the generation from the pixel space to a lower dimensional latent space. We show the effectiveness and scalability of the proposed method via extensive experiments, where the proposed method achieves a significantly better privacy-utility trade-off compared to existing alternatives. Notably, our method the first DPGM to scale high-resolution image sets (up to 256 256). Our code is available at
1 Introduction
Large-scale datasets (Deng et al., 2009: Lewis et al. 2004: Bennett et al. 2007] facilitate the great success of modern machine learning (ML) systems. However, privacy concerns arise especially when sensitive data (e.g. human face images and medical data) are involved in the training. Prior privacy-preserving techniques include naive data anonymization (Narayanan & Shmatikov] 2008), k-anonymity (Sweeney 2002). I-diversity
Machanavajjhaaa et al. 2007 semantic security (Goldwasser & Micali, 1984), and differential privacy (DP) [work 2006), where DP is recognized as a rigorous quantization of privacy, and it has become the gold- standard privacy-preserving technique in the current ML community.
Differentially private generative model (DPGM) aims synthesize data that are distributionally similar the private data while satisfying DP guarantees so that the individual privacy that can be inferred from the incremental change in the dataset is bounded. DPGM can (1) serve a proxy for releasing private data, and can (2) generate data for private data analysis tasks (e.g. data querying and MI. tasks) without incurring further privacy cost, as ensured by the post-processing theorem (Dwork et al. 2014).
Generative adversarial network (GAN) [Goodfellow et al.) 2014h has attracted the most attention in de- veloping DPGMs (Xie et al.] 2018
Augenstein et al. 2020 Chen Torkzadehmahani et al. 2019 Jordon al., 2019: Long et al. 2021
2020), and VAE also draws lot of attention (Chen et al.] 2018 8 Acs
al.) 2018 Takahashi al. 2020
Pfizzner & Arnrich, 2022
Weggenmann et al. 2022
Bernau al. 2022

---

Published in Transactions on Machine Learning Research (10/2023)
In contrast, the DPGMs based on the normalizing flow are relatively limited (Waites & Cummings 2021 Lee et al. 2022). Due to the capability of the exact density computation, flow models are particularly useful when density is of interest in some applications. For example, developing a DP-flow could benefit performing anomaly dctection a privacy-preserving manner (Waites & Cummings, 2021 However, both existing DP-flow works (Waites & Cummings 2021 Lee et al. 2022 are restricttdd to labular dalasets (with
lower data dimensions). It remains unclear whether DP-flow is feasible on higher-dimensional datasots such as image datasets.
Abadi et al. (2016) propose Differentially Private Stochastic Gradient Descent (DP-SGD) algorithm, which has become the standard approach to train DP deep learner. The core steps of DP-SGD are clipping the per-example gradient norm and injecting Calibrated Gaussian noise into the aggregated gradient of a batch. Briefly, let denote the model dimension, the noise introduced by DP-SGD in each iteration is given by ~ NV,, T where C' the clipping bound of gradient norm and a a noise multiplier. Apparently, B||l po² xc p, which means the model utility may not be preserved with DP-SGD for large model. For example, Yu et. al. (2021) show that the gradient will be submerged in the added noise (by DP-SGD) when the model becomes larger drop could become more pronounced under strong privacy guarantecs (e.g. c= 1), as the noise multiplier becomes larger. Note that according to e-DP definition (given in Section 2 the privacy protection becomes weak when 6 = 10 or above, because the ratio of two probabilities is upper bounded by exp(10) = 2.2 x 10", whereas they are presumed to be comparable for practical deployment (e.g. c <= or below). However, a line of recent SoTA DPGMs only generate acceptable images at = 10, whereas they are not able to scale
series of ResNet (He et al. 2016) variants. This utility
well on small (e.g. 6 6=1) (Chen al. 2020 Jordon et al. 2019 Torkzadehmahani al. 2019).
Training& normalizing flow with DP-SGD searcher/engiin /engineer will encounter the two non-trivial empirical challenges, as also pointed out by Lee et al. (2022): (1) batch normalization (BN) challenge: flow models usually apply batch normalization (BN) layer in each block to boost the performance, where the per-cxample gradicnt (in DP-SGD) is not available, as BN will break the independence among all gradients models generally consist of repetitive blocks of invertible transformations with large depth, which results in higher model complexity compared to other generative models. We explore the challenges with two SoTA flow models, i.c. RealNVP (Dinh et al. 2017) and Glow (Kingma & Dhariwal normalization as an alternative to BN, thus per-example gradient is computable (i.e. no BN challenge for Glow). However, both RealNVP and Glow suffer from the model complexity challenge, i.e. they tend to be more complex than other generative model counterparts for synthesizing images with similar fidelity, thus are more challenging to attain desirable model utility with DP-SGD. Our motivating example is shown Figure G where naively training a flow generative model in the pixel space with DP-SGD results in null model utility.
image set seemss ostensibly easy, but any ML re-
a batch; (2) model complerity challenge: flow
2018). Glow uses activation
Our contributions can be summarized as follows:
We explore the practical challenges of training a DP flow via DP-SGD on image sets, and propose an efficient and effective solution, i.e. differentially private latent flow (DP-LFlow), by reducing the training of flow from the full pixel space to a lower-dimensional latent space, which is more resilient to the noise perturbation (by DP-SGD).
Training DPGM on high-resolution images (256 x 256 pixels and beyond) is extraordinarily chal- lenging due to the inevitably increased model dimension, and to our best knowledge, none of the existing related works attempt to do so. In this work, we will show that DP-LFlow is also scalable to the high-resolution image (256 x 256) generation with DP constraints.
The proposed method yields state-of-tho-art (SoTA) performance on model utility under the same (c. 8)-DP constraint on widely compared image benchmarks. Morcover, our method indicates more robustness and scalability different datasets (gray-scale and RGB) and different DP constraints.
2

---

Published in Transactions on Machine Learning Research (10/2023)
Algorithm 1: Gradient perturbation in DP-SGD
Input: Private training set X = {x loss function L(),, batch size B, noise multiplier gradient
clipping bound C, model parameter
for i - to B do
(x(): Vx[(x;;;)
go(x;) = go (x;) min 11 (x)
E E (xx) NN(0,
90
2
Preliminary
In this section, recall background knowledge in differential privacy.
2.1 Differential Privacy
Differential privacy is widely regarded as a rigorous quant ization of privacy, which upper-bounds the deviation in the output distribution of randomized algorithm given an incremental deviation in the input. Formally, have the following definition:
Definition 2.1 ((c, 6)-DP (Dwork al. 2014 ). A randomized mechanism M D with domain D and range R satisfies (e, differrretl privacy if for any two adjacent inputs D, D' E and for any subset of outputs S R it holds that
Pr(M(D) S) S exp(c) PMMM))Pr(((()) +
(1)
where adjacent inputs (a.k.a. neighbouring datasets) only differ in one entry. Particularly, when 0, we say that M e-D..
There is a convenient parallel composition theorem for eDP mechanisms:
Theorem 2.1 (Parallel composition theorem of e-DP, (McSherry 2009)). Let M; 101-1,2,, 1, k) be k DP mechanisms, and each M; satisfies DP. Given deterministic partitioning function f. let D,, D2...... Dj be the disjoint partitions by executing f on D. Releasing (D1),,.... (D(()....MM(())) satisfies max;e (1.2..... DP.
We will extend the above parallel composition to the 5c notion in Scction 3
A famous theorem, iee post-processing theorem, which utilized by existing works (as well as ours) for proving DP guarantee of published model, is given by:
Theorem 2.2 (Post-processing theorem, (Dwork et al. 2014 If M satisfies (e,0)-DP, F M will satisfy (e, (eP for any function with o denoting the composition operalor.
Sampling from DPGM is independent of training data, thus can be viewed as post-processing step and does not breach the DP guarantee.
Rényi differential privacy (RDP) extends ordinary DP using Rényi's a divergence (Rényi 1961 and provides tighter and easier composition property than the ordinary DP notion, thus we adopt RDP accumulate the privacy cost. We defer details of RDP and implementation to Appendix E
2.2 DP-SGD
Within predetermined training iterations, in each iteration, DP-SGD (Abad et. al., 2016 subsamples a batch from the training set, clips and perturbs the gradient as in Algorithm [ and optimizes the model with privatized gradient Zg. As mentioned earlier, the norm the Gaussian noise introduced line Algorithm will scale lincarly with the model dimension, thus will generally degrade the utility of large models.
3

---

Published in Transactions on Machine Learning Research (10/2023)
2.3 Flow-based Generative Models
We brielly recap the flow generative models. Flow models learn a bijective map T between a simple prior distribution Qo (e.g. Gaussian) and the target distribution qi z + T(z) ~ Through the change-of- variable formula, the log-likelihood of input is tractable:
dz
dT (x)
(T(x) = 20(x)) det
dx - 2o (T (x)) det
(2)
dx
Parameterize the bijective map by neural networks, we can train flow models by minimizing the Kull- back-Leibler (KL) divergence between the true and estimated distribution:
= -min / ( log (x(x)
g(x) dx
min (xx
(3)
= =min T -x- (x) - H[g
(4)
where [[[g] is the entropy of true distribution. Therefore, training the flow model amounts to minimizing the negative log-likelihood of input.
3 Method: DP-LFlow
As shown in Yu et al. 2021), a smaller/simpler ResNet is more resilient to DP-SGD. The intuition is that the model utility will saturate as the model complexity increases. Therefore, an over-complicated model may (unnecessarilyy be disturbed by more noises in DP-SGD. We first confirm this insight for DPGMs as well. Consider privately training a VAE on MNIST (LeCun et al. [9988 for example. Figure shows such comparison under non-private and private settings (with different privacy costs), respectively, where the generation quality is measured by Fréchet Inception Distance (FID) (Heusel et al.) 2017) (lower is better). indicates that a smaller VAE generates better images than larger counterparts (both qualitatively
Figure ! and quantitatively) under the DP training, even though larger VAEs perform better in the non-DP setting, In fact, shrinking the model size under the DP training will benefit from the following aspects: (1) smaller models are more resilient to (larger) noises (associated with strong DP guarantees); (2) a significant training time overhead remains a notable challenge for DP-SGD (Subramani et al. 2021) due to the gradient clipping and randomization. Smaller models could facilitate more efficient DP training; (3) as suggested by prior work (De et al. 2022), larger batch size in DP-SGD contributes to better model utility. With limited GPU memory, smaller models allow for large training batch size, since both input and model are sent to GPU for training in practice.
However, the model expressiveness will be restricted if the model is too simple. Therefore, we aim to design a model that small yet expressive enough so that we can achieve a better privacy-utility trade-off with DP-SGD. Inspired by the recent latent diffusion model (Rombach et al.) 2022 that achieves SoTA text-to-image generation performance
Latent Flow:
reducing the diffusion process from the raw input space to a lower dimensional latent space, we propose to train normalizing flow in similar manner (i.e. on the latent code produced by an autoencoder), by simultaneously minimizing the reconstruction loss of the autoencoder and the negative log-likelihood of the flow. Specifically, let σ, 0,w parameterize the encoder, decoder, and
flow, respectively, and we adopt the notations in Section 2.3 Let w = fx( be the latent embedding of the autoencoder, the training objective can be written as:
min F. Ex [|x D x (oog log det dT (w) 0,8.0
dw
Autoencoder losss
Normalizing flow loss
where z ~ 9o = 1), and T is a temperature parameter. As shown in Rombach et al. (2022), the semantic meaning of most images still remains after aggressive compression, thus allowing us train flow in an aggressively trimmed latent space, which avoids unnecessary and expensive computation on full input dimensions. It. is worth mentioning that latent flow is also not sensitive to BN layers (e.g. for RealNVP), iee.
4

---

/ Published in Transactions on Machine Learning Research (10/2023)
3
6 6
12
9
'
5
3
0
1
&
4 7
2
2
3
9
0
9
2
4
6
7
2
g 9 9 6 9
0
0
4
6
9
5
5
65
5
7
Z
0
7 7
4
5
4
8 P 8
3
0 0 P
3
7
2
f
3
5
9
8
2
8 8
1
5
3
3
4
5
24
4
C
2
6
6
7 7
9
2
2
J
3 4
7
6
17
6
9
5
0
6
2
5
S
8
O
1
5
3
4
8
4
7
A
6
5
200
non-DP
176
DP (eps=10)
(epse1)
125
100
50
26
0.54
1.8
19.8
75.5
Modeisize (MB)
3
7
9
9
2
E /
7,
9
8
6
8
3
0
5
7
8
9
4
7
g
0
7
4
7
y
7
2
6
7
0
/
A
4
6
9
0
0
5
2
3
5
0
3
8
5
6
7
0
0
4
9
9
0
1
2
5
4
&
6
8
0
f
2
3
4
5
6
7
9
0
Q
2
3
6
7
8
5
G
1
3
4
5
6
8
9
3
4
5
9
9
0
2
4
5
7
8
9
6
I
3
4
7
6
0
7
1
9
5
0
(
7
5
7
4
0
5
3
4
4
7
6
9
5
3
0
2
7
Figure 1: FID vs. VAE size (in MB). We only vary the model complexity, with all the rest training parameters (c.g. subsampling rate, noise multiplier, training iterations) fixed.
the utility of latent flow is slightly reduced by removing the BN layer, which validates the use of DP-SGD for latent flow models. Under the non-private setting, the proposed model with configuration of size 2.5 MB achieves an FID 12.5 MNIST, which is already superior to VAE counterparts with larger model size (e.g. in Figure 1. suggesting its promising usage under the private training.
Partitioning Datasct: Current SOTA methods tend to apply conditional generative models (conditioned on labels), where the label is encoded in the model as part of the input, thus the noise perturbation also distorts the label information, which is unnecessary. We run DP-SGD on the proposed model under the con- ditional setting (autoencoder + conditional flow), and observe that the label information is largely distorted when = 1 (sce Section 4.5 for dotails). To circumvent the perturbation labels, we propose to partition the dalasel according lo labels, train unconditional generative models on each of the subsets, and release the union of all unconditional generators the resulting model. The partitioning is also beneficial for shrinking the model size, as each generator is only interactive with a sole data modality instead of multi-modalities. Adapted from proposition 2.5 in [i et al. 2016), the DP guarantee for the union can be derived from the parallel composition, by extending the original parallel composition theorem (Theorem 2.1 from c-DP notion to (e, oD notion. The proof can be found in Appendix A
Theorem 3.1. Let Mj 11 ..2.....,k) k) be DP mechanisms, and each M; satisfies (c, &DDDP. Given a deterministic partitioning function f, let D1, D2...., D3 be the disjoint partitions by executing f on D. Releasing
satisfies (1..,.(112..... E,, maxxg(1.......)) 6
For simplicity, we set 6, 5, the same as the target e, 5 for all sub-models in the experiment. The schematic workflow of DP-L.llow is shown in Figure 7
5

---

Published in Transactions on Machine Learning Research (10/2023)
4 Experiments
In this section, evaluate and compare DP-LFlow against SoTA basclines through extensive experiments
in Section 4.2. More importantly will show that DP-LFlow is amenable to high-resolution image sets
in Section 4.3 which was hardly studied in prior related works. Besides, since flow models are capable of computing the exact likelihood of input, they are sometimes more applicable when donsity estimation is of concern. Section 4.4 shows such an example, where DP-LFlow can delect intra-dalaset anomaly with DP guarantees. NVP used as the flow model DP-LFlow, as yields better performance practice. Implementation details and neural network configurations are given in Appendix B
4.1 Experimental Setup
Datasets: We consider three widely used image datasets, including both grayscale images (MNIST (Leunn et al. 1998), Fashion MNIST (Xiao et. al., 2017)) and RGB images (CelebA (Liu et al. 2015 ) as well as one high-resoluting RGB datasets (CelebA-HHe HQ (Karras et al. 2018), : for our presentation only). For MNIST and Fashion MNIST, we generate images conditioned on 10 respective labels. For CelebA and CclcbA-HQ. we condition on gender. Descriptions and preprocessing of the datasets given in the Appendix B
Evaluation Tasks & Metrics: We evaluate and compare DPGMs by two metrics via 60k generated images:
Fréchet Inception Distance (FID) (Heusel et al. 2017
Classification accuracy. We train three different classifiers, e.g. logistic regression (LR), multi-layer perceptron (MLP), and convolutional neural network (CNN), on generated images, then test the classifier on real images. We take runs and report the average. See Appendix for details of classifiers.
Post-processing: We observe that FID is quite sensitive to the noisc around the object (o.g. digits, cloths) and the sharpness of images, even when the semantic content is retained. We apply a post-processing step to our DP image generation by first smoothing and then sharpening the generated images, which could lead to 10 improvement in FID. Whereas this post-procee post-processing does not improve classification accuracy too much. Adjusting the blurriness and sharpness of generation not involved with training data, thus considered a post-processing step that does not breach the DP guarantee by Theorem 2.2 We will report our quantitative evaluations with and without this trick.
SoTA Baselines:
Our method is compared with the following baseline methods that also developed
image datasets, Le. DP-CGAN (Torkzadehmmaaani et al. 2019 DP-MERF (Harder et al. 2021 Datalens
(Wang et al. 2021). PATE-GAN (Jordon et al., 2019). G-PATE (Long et al. 2021), GS-WGAN (Chen et al. 2020), SSnnknooor (Cao et al. 2021h. For more details, we refer readers to Section 6 and respective
references.
4.2 Comparison with SoTA Baselines
We follow the general benchmark for evaluating DP generation, by considering both weak (e.g. 10) and strong (e.g. 1) privacy guarantees. The proposed DP-LFlow is compared with SoTA baselines through extensive qualitative and quantitative experiments on both grayscale and RGB image datasets. For completeness, we also report our quantitative results without the post-processing trick, where the relative rank of our method without the post- prst-procsssing trick remains the same as with the trick.
4.2.1 Under the Weak DP Guarantee e 10
Most existing works perform reasonably when 10, as shown in Figure 2 Nevertheless, as shown in Table DP-L.Flow achieves significant improvement in both FID and classification accuracy.

---

Published in Transactions on Machine Learning Research (10/2023)
Table 1: Quantitative comparison on MNIST and Fashion MNIST given (10, 10 (10 Acc denotes classi- fication accuracy, which shown in %. 1 and & refer higher is better or lower is better, respectively, We use boldface for the best performance. Results of DP-CGAN, GS-WGAN, DP-Sinkhorn are cited from Cao et al. 2021 Results of G-PATE and DataLens cited from their papers, respectively. Ours denotes result without the post-processing trick.
MNIST
Fashion MNIST
Method
FID 4
LR
MLP
CNN
FID
LR
MLP
CNN
Acc f
Acc
Acc f
Acc f
Acc t
Acc
Real data (non-private)
1.6
92.2
97.5
99.3
2.5
84.5
88.2
90.8
DP-CGAN
179.2
63
243.8
51
50
DP-MERF
121.4
79.1
81.1
82.0
110.4
72.3
70.8
73.2
G-PATE
150.6
80.9
171.9
69.3
DataLens
173.5
80.7
167.7
70.6
GS-WGAN
61.3
79
79
80
131.3
68
65
65
DP-Sinkhorn
55.6
79.1
79.2
79.1
129.4
70.2
70.2
68.9
Ours
16.8
85.8
93.0
95.3
72.3
78.6
78.8
81.0
Ours
25.4
85.1
92.4
94.8
80.8
78.1
78.4
80.7
/
3
4
3
5
7
$
DP-CGAN
66
3
4
6
7
$
O
J
2 7
3
4
A
V /
7 7
8 8
9 9
DP-MERF
3
I
3
7
G-PATE
6
2
3
4
5
6 G
DataLens
0
3
5
6
9
O 2
3
4
5
6
7
8 8
9 9
GS-WGAN
O
/ /
2.
3
4
5
6
7
0
3
4
5
G
7 7
8
DP-Sinkhorn
0
3
4
!
L
S
9
0
/
3
4
5
6
7
8 8
9 9
0
1
2
3
4
5
7
Ours
0
/
3
4
5
6
7
8
0
2
3
5
7
4
9
Figure 2: Qualitative comparison on MNIST and Fashion MNIST under (10, 10 10-1100---- SD-DP. Images of DP-CGAN, GS-WGAN, DP-Sinkhorn are cited from (Cao et al. 2021). Images of G-PATE and DataLens cited from their papers, respectively.
4.2.2 Under the Strong DP Guarantee c=]
When 6 <= 1, i.c the more challenging case under the DP setting, we consider both gray-scale image datascts and RGB image datasets for comparison. Figure and Figure show that the existing works are not readily amenable to small such as 1. In contrast, DP-LFlow exhibits significant visual improvement all three datasets.
Quantitatively, Table indietees indicates that DP-LFlow outperforms other baselines on CNN classification accuracy on all three datasets. Besides, DP-LFlow achieves FID scores that are comparable to SoTA performance on Fashion MNIST and achieves the best FID on MNIST and CelebA. Given that the three datascts are either largely different in input dimension or largely different in semantic content, the superior performance of DP-L.llow further validates its generality and versatility.

---

Published in Transactions on Machine Learning Research (10/2023)
Q
3
3 7
8
DP-MERF
2
6
G-PATE
DataLens
0
! 2
?
DP-Sinkhorn
0
1
3 3
7
1
9
5
GS-WGAN
3
0
/ /
2
4
5
8
8
0
2.
3
4
5
6
7
8
Ours
O
3
4
5
6
8
9
0
A
3
4
5
6
7
8
9
0
1
3
5
6
Figure 3: Qualitative comparison on MNIST and Fashion MNIST under (1, 10- Images of G-PATE and DataLens are cited from their papers, respectively.
DataLens
DP-MERF
Ours
Figure 4: Qualitative comparison on CelebA conditioned on gender under (1, 50 Images of DataLens are cited from the original paper. Top row: female. Bottom row: male.
It worth mentioning that couple of recent works on DP diffusion models are also capable of generating realistic images under DP constraints and indicate superiority on downstream ML lasks (Dockhorn et al.
2023 Ghalebikesabi al. 2023). For example, when 1, Dockhorn et al. (2023) achieve 93.4 and 73.6 test classification accuracies on MNIST and FMNIST, respectively, while ours are 88.9 and 77.4. However, their required computational resource is significantly higher than DP-LFlow, e.g. Dockhorn et al. (2023) need 8 GPUs and one day to train a DP diffusion model on MNIST and FMNIST, while our method only requires 1 single GPU and a few hours; On CelebA, Dockhorn et al. 2023 needs 8 GPUs and days, while our method only needs GPU and around half-day. More importantly, DP-LFlow still has two unique properties compared to DP diffusion models: (1) DP-LFlow can generate large images up to 256 256
(sce Section while Dockhorn et al. (202) Ghalebikesabi et al. 2023 only present generation results
images up to 3 32 32; (2) DP-LFIe low is able to conduct the DP- DPOOO detection task (see Section 4.4
4.3 Generating High-resolution Images with DP Constraints
To our best knowledge, the highest resolution image dataset used related works is CelebA downsampled at 64 pixels (by DataLens and G-PATE), which, however, is still not typically considered a high-resolution space. Here we consider a high-resolution dataset CelebA-HQ in 256 x 256, which is also close many modern large-scale image classification benchmark datasets such as ImageNet. (224 x 224 256 x 256).
8

---

Published in Transactions on Machine Learning Research (10/2023)
Table 2: Quantitative comparison on image datasets given (1, 10110-)) DDDDP FMNIST denotes Fashion MNIST. Ace denotes CNN classification accuracy, which is shown in %. 1 and refer higher is better or lower is better, respectively. We use boldface for the best performance. Results of PATE-GAN and G-PATE are cited from Long et al. (2021). Results of DataLens cited from Wang et al. 20211) Ours* denotes result without the post-processing trick.
Dataset
Metrics
PATE-GAN
DP-MERF
DP-Sinkhorn G-PATE
DataLens
Ours
Ours *
FID +
231.5
118.3
168.5
154.3
186.1
69.7
83.4
MNIST
Acc
41.7
80.5
60.5
58.8
71.2
88.9
88.2
FID 4
253.2
104.2
184.2
214.8
195.0
132.4
143.5
FMNIST
Acc
42.22
73.1
62.1
58.12
64.8
77.4
76.8
FID 4
434.5
219.4
293.2
297.7
204.8
217.7
CelebA
Acc
44.48
57.6
70.2
70.6
73.2
72.1
Figure 5: Samples from DP-LFlow trained on CelebA-HQ under (10, 10 DDP Top 2 rows: female. Bottom rows: male. FID = 328.6, LR classification accuracy = 77.4.
Note that now the input dimonsion drastically increases from x 28 x 28 = 784 (for MNIST and Fashion MNIST) 3 x 256 x 256 1.97 x 10 This means that the model generally has to drastically scale up adequately learn the input distribution, leading to a significant challenge for training a DPGM with DP-SGD.
Nevertheless, with the help of lower dimensional latent space where while retaining the semantic content, we can restrict the generative model to size (~ 22 MB, compared to ~ 2.5 MB on MNIST) that is suitable for DP training yet adequate to learn the data distribution. By training a DP-oFllo on Cel,bA-HQ, Figure 5 shows that DP-LFlow is able to produce diverse and recognizable face images with DP constraints on such high dimensional input space.
apply aggressive compression
4.4 Differentially Private Out-of-distributio (OOD) Detection
One exceptional feature of flow models is their capability of ezact density ostimation, which can benefit certain applications such as onl-of-distributtion delection. A natural method to do so is to thresholding the log-likelihoode since it is presumed that the in-distribution (InD) data (where the training data are drawn from) will be assigned a higher likelihood than OOD data by the flow model. This idea can be readily extended to latent flow models by checking the likellood of latent code mapped from the input data. As sub-models privately trained each subset by class, immediately conduct the differentially

---

Published in Transactions on Machine Learning Research (10/2023)
Table 3: Differentially private OOD detection results. The evaluation metric shown by AU- ROC (higher is better, and 1.0 means InD and OOD likelihood are perfectly separable).
a
MNIST
Fashion MNIST
InD class
10
e
6
10
E
(b)
0
0.98
0.87
0.90
0.87
1
0.99
0.98
0.97
0.96
2
0.86
0.75
0.90
0.88
(c)
3
0.88
0.82
0.91
0.91
4
0.95
0.85
0.87
0.85
5
0.90
0.75
0.91
0.86
Figure 6: Training a RealNVP (~150 MegaBytes (MB)
0.97
0.77
0.84
0.82
CelebA (a) non-privately with BN layers, (b)
7
0.93
0.88
0.97
0.97
privately without BN layers, (c) under 510
8
0.89
0.74
0.86
0.79
without BN layers
9
0.95
0.83
0.94
0.93
Average
0.93
0.82
0.91
0.88
private intra-dataset OOD detection, by treating each training class as the InD and the rest classes combined OOD. A lower log-likelihodd of the latent code computed from input data implies OOD-ness. We use Area Under Receiver Operating Curve (AUROC) the evaluation metrie, which is threshold-fre and widely used in OOD detection literature. Table 3 indicates that under weak and strong guarantees, DP-LFlow can effectively detect intra-dataset OOD input on both MNIST and Fashion MNIST across all Inl) classes in a privacy-prescrving manner.
4.5
Ablation Study
DP-LFlow contains two main components, and we conduct ablation study follows.
4.5.1 Flow in the Pixel Space Latent Space
We first trained a RealNVP with DP-SGD by removing the BN layer. Figure 6 show that the generation from a DP-ReaINVP (subfigure (c)) is largely submerged in the noise. Besides, we also trained a Glow with DP-SGD (with various model complexities), yet ended up with null synthesis when targeting (10, 10-3) Both RealNVP and Glow suffer from the model complexity challenge. Compared to the generation of DP- LFlow in Figure 4 (which even targets stronger privacy regime, i.c. = 1), training flow in the latent space relieves the model complexity challenge, thus is more resilient to the noise perturbation introduced by DP-SGD.
4.5.2 With vs. Without Partitioning the Dataset
Qualitatively, we observe that training a conditional DP-L.Flow on the whole dataset will fail to correctly condition on cach label, (espocially under c c= 1), i.. the gencrations conditioned on different labels are not consistent with the true labels, thus the numerical classification accuracy is relatively lower, although FID is fairly comparable. We summarize the numerical comparison in Table
The reason is that, in contrast to uninvertible models like conditional GAN and conditional VAE that only encode labels in the latent vector, the label information in the conditional flow is encoded in all coupling layers, because flow is an invertible model so that both forward and backward pass needs label information. Thus, training conditional flow with DP-SGD will distort the label information more than uninvertible models. We also tried modify the way of encoding labels in different variants: only encoding labels in the first, last, both first and last coupling layers, or only encoding labels in the latent vector, but none
10

---

Published in Transactions on Machine Learning Research (10/2023)
Table 4: Ablation on partitioning the MNIST dataset. We compare the performance without post post-processing
tricks (smoothing and sharpening) for simplicity.
FID
LR Acc t
MLP Acc t
CNN Acc 1
With partitioning
10
25.4
85.2
92.7
95.0
Without partitioning
10
21.3
86.2
91.5
91.8
With partitioning
83.4
83.2
82.4
88.3
Without partitioning
87.1
40.8
72.5
77.5
of the variants solve the issue. Instead, resorting to parlitioning the dataset can readily bypass the label distortion in the private training, which leads to better model utility compared to single conditional model on all classes, and by Theorem 3.1 we can retain the same level of privacy guarantees as publishing a single conditional generator. In other words, partitioning the dataset contributes to improving the privacy-utility trade-off.
5 Discussion
Pretraining the autoencoder: Though our current training proceeds by simultaneously updating the autocncoder and flow, one may wonder whether we can pretrain an autoencoder first and then train flow with the pretrained autoencoder. However, there are a few concerns that impede the two-stage training paradigm:
Pretraining on a public dataset: The majority of existing works (including all our compared works) do not use external public datasct for pretraining, so would make the comparison unfair if we do so. Besides, we need to put a strong assumption that the public dataset does not have any privacy concerns. While this can be easily ensured in experiments, it may not always be true in reality, thus is likely to undermine the potential impact of this work. Therefore, we stick to restricting our information to the private dataset only.
Pretraining on the private dataset: In this case, changing point in the dataset will result in a different pretrained autoencoder, which means all the gradients in batch (when running DP- gradient is affected in original DP-SGD, thus the
SGD on flow) will change, instead of at most sensitivity of the gradient will scale up by B, Lee the batch size. To make the same DP guarantee hold, we can recalculate the by replacing the original noise multiplier a with α B to account for the gradient sensitivity change in a minibatch. However, the actual noise multiplier is much larger and will undoubtedly destroy the model utility.
Batch normalization (BN) vs. Group normalization (GN): Replacing BN with GN is common practice for training DP models, e.g. as in Luo et al. (2021). However, unlike the image classification network [Luo et al. 2021) that only takes forward pass, flow generative models are invertible models, and any normalization technique will run into issues when we want a backward pass (e.g. sampling), i.e. there is no mean and variance for unnormalization. Currently, BN layer in flow models tends to register two buffers, i.e. running mean and running variance, to keep track of the batch mean and batch variance in the training, then fix it to generate data in the backward pass. We implement GN layer for flow models similarly, i.c. registering two bullers to keep track of the group mean and variance in the training (we use 4 groups for each instance). However, during generation, we first try repeating the group mean and variance for the sampled batch, but the generation exhibits no variance, i.c. they all look similar. Then we change the registered buffers to keep track of a pool of group mean and variance, but the generation becomes null. We leave a tailored normalization layer that is suitable for private training for future work.
The insight behind DP-LFlow, ie. shrinking the model size, seems to contradict the intuition and the findings in a prior study on scaling laws of language models (Kaplan et. al. 2020 because
Limitation:
11

---

Published in Transactions on Machine Learning Research (10/2023)
model parameters generally mean better learning capability. However, Kaplan et al. (2020) conducted all experiments in the non-private setting, and our findings in Figure 1 on image generators are consistent with Kaplan et al. (2020) in the non-private setting. In the private setting, Figure | motivates us to improve the performance of DPGMs by preventing the model from being over-complicated. Nevertheless, it is apparent that an over-simple generalive model (certainly including DP-LFlow) is not able to generate realistic images either with or without DP guarantee. Therefore, shrinking the model size may impose an upper bound on the model utility.
6 Related Work
We categorize related works by approaches:
The vast majority of related works are realized by training different generative models with DP- SGD. GAN: DP-GAN (Xie et al. 2018 first trains GAN with DP-SGD algorithm, where the discriminator is trained with DP-SGD, then the generator is automatically DP as ensured by post-processing thcorem. DP-CGAN (Torkzadehmahani el al.) 2019) extends the idea into the conditional setting. VAE: DP- VaeGM Chen et al. 2018) trains l VAFs on classes of private data with the DP-SGD algorithm, and returns the union as generations. This work only focuses on privacy attacks. DP-kVAE (Acs al. 2018 first partitions the dataset into k clusters by kernel k-means method, then trains & VAEs on each data cluster with DP-SGD. However, their generation exhibits clear mode-collapse. Flow: DP-NF (Waites & Cummings) 2021 directly trains a flw-based model by DP-SGD. DP-HFlow (Lee et al. 2022 designs a fine- fine-grained gradient clipping strategy to increase the signal-to-noise ratio and accelerate the training. However, both works relating to flow models are limited (low dimensional) tabular datasets. Diffusion model: A few recent papers also tried training the powerful diffusion models with DP-SGD (Dockhorn et al. 2023 Ghalebikesabi et al., 2023), which can generate realistic images under DP constraints. However, their training is significantly more computationninteeniive,, e.g. Dockhorn et al. (2023) requires 8 GPUs and one day to train a DP diffussion model on MNIST and FMNIST.
DP-SGD:
PATE Mechanism:
Private Aggregation of Teacher Ensembles (PATE) Paaperot et al. 2017 2018 is
another mechanism for learning a DP model, by perturbing the aggregated information from an ensemble of teacher models with noise. PATE-GAN (Jordon et al.) 2019) first applics PATE mechanism to GAN, where the discriminator becomes non-differentiable, thus a student discriminator is trained with all teacher ensembles, which is then used train the generator. G-PATE (Long al.) (2021) sanitizes the aggregated gradients from teacher discriminators to the generator to make the generator DP. However, gradient vectors need to be discretized in each dimension to employ the PATE mechanism that only takes categorical dala as input. DataLens (Wang et al.) 2021) further improves G-PATE by introducing a three-step gradient compression and aggregation algorithm called TopAgg.
Kernel-based Methods: DP-MERF HHrder et al. 2021 proposes to perturb the kernel mean embed- dings of real data through random Fourier features, and train a generator by minimizing the maximum mean discrepancy (MMD) between the noisy embedding of private input and embedding of generation. PEARL [Liew al. 2022) extends the idea of DP-MERF by introducing an adversarial objective on sampling frequencies, which indicates improvement performance.
Others: GS-WGAN (Chen et al. 2020 creates a privacy barrier the output of the generator of a Wasserstein GAN (WGAN), based on the observation that only the generator will be published, thus only the generator needs to be private. DP-Sinkhorn (Cao et al. 2021 adds privacy barrier in similar way GS- WGAN, where a Sinkhorn loss is used as the training objective.
7 Conclusion
Though DP-SGD currently the workhorse algorithm for training a deep learning model, remains a big challenge whether it can be applied to training large models with acceptable model utility. In this paper,
12

---

Published in Transactions on Machine Learning Research (10/2023)
first show that training DP flow via DP-SGD is highly challenging (or even infeasible) with achieving acceptable utility due to a few particular challenges of flow models, and then propose effective solution, l.e DP-LF,lo, by reducing the flow training from the full input space to a lower dimensional latent space, so that the model is more resilient to (larger) noise perturbation introduced by DP-SGD. Experimental results on widely compared image benchmarks remonstrate the generality and scalability of DP-LFlow on different image spaces (grayscale and RGB) and different DP constraints (weak and strong DP guarantees). Notably, to our best knowledge, DP-LFlow the first DPGM to scale to high-resolution image datasets, which further validates its effectiveness and versatility.
Acknowledgments
We thank all the reviewers and the action editor for thoughtful comments. We gratefully acknowledge funding support from NSERC and University of Waterloo, and the computing resources from the Vector Institute.
References
Martin Abadi, Andy Chu, Ian Goodfellow, H. Brendan McMahan, llya Mironov, Kunal Talwar, and Li Zhang, Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp. 308 318, 2016.
Gergely Acs, Luca Melis, Claude Castelluccia, and Emiliano De Cristofaro. Differentially private mixture generative neural networks. IEEE Transactions on Knowledge and Data Engineering, 31(6):1109 1121, 2018.
Sean Augenstein, H. Brendan McMahan, Daniel Ramage, Swaroop Ramaswamy, Peter Kairouz, Mingging Chen, Rajiv Mathews, and Blaise Aguera y Arcas. Generative models for effective ml private, decen- tralized datasets. In International Conference on Learning Representations, 2020.
James Bennett, Stan Lanning, et al. The Netflix prize. In Proceedings of KDD cup and workshop, pp. 35, 2007.
Daniel Bernau, Jonas Robl, and Florian Kerschbaum. Assessing differentially private variational autoen- coders under membership inference. arXiv:2204.07877 2022.
Tianshi Cao, Alex Bie, Arash Vahdat, Sanja Fidler, and Karsten Kreis. Don't generate me: Training differentially private generative models with Sinkhorn divergence. In Advances in Neural Information Processing Systems 21, 2021.
Dingfan Chen, Tribhuvanesh Orekondy, and Mario Fritz. GS-WGAN: A gradient-sanitized approach for learning differentially private generators. In Advances in Neural Information Processing Systems, pp. 2020.
Qingrong Chen, Chong Xiang, Minhui Xue, Bo Li, Nikita Borisov, Dali Kaarfar, and Haojin Zhu. Differen- tially private data generative models. arxiv:1812.0227, 2018
Soham De, Leonard Berrada, Jamic Hayes, Samuel L Smith, and Borja Balle. Unlocking high-accuracy differentially private image classification through scale. arXiv:2204.13650, 2022.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. JEFE conference on computer vision and pattern recognition, pp. 248-244822 255, 2009.
Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. Density estimation using real nvp. In International Conference on Leurning Representations, 2017.
Tim Dockhorn, Tianshi Cao, Arash Vahdat, and Karsten Kreis. Differentially private diffusion models. Transactions on Machine Learning Research, 2023.
13

---

Published in Transactions on Machine Learning Research (10/2023)
Cynthia Dwork. Differential privacy. In International Colloquium Automata, Languages, and Program- ming, pp. 1-12, 2006.
Cynthia Dwork, Aaron Roth, et al. The algorithmic foundations of differential privacy. Found. Trends Theor. Comput. Sci. 9(3-4):211-407 2014.
Sahra Ghalebikesabi, Leonard Berrada, Sven Gowal, Ira Ktena, Robert Stanforth, Jamic Hayes, Soham De, Samuel Smith, Olivia Wiles, and Borja Balle. Differentially private diffusion models generate useful synthetic images. arXiv:2302.13861 2023.
Shafi Goldwasser and Silvio Micali. Probabilistic encryption. Journal of computer and system sciences, 28 (2):270-299) 299, 1984.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural information processing systems, volume 27, 2014.
Frederik Harder, Kamil Adamczewski, and Mijung Park. DP-MERF: Differentially private mcan embeddings with random features for practical privacy-preserving data generation. In International Conference on Artificial Intelligence and Statistics, pp. 1819 1819-1827, 2021.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition (CVPR), 2016.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs trained by a two time-scale update rule converge to local Nash equilibrium. In Advances in Neural Information Processing Systems, volume 30, 2017.
James Jordon, Jinsung Yoon, and Mihacla Van Der Schaar. PATE-GAN: Generating synthetic data with differential privacy guarantees. In International conference on learning representations, 2019.
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv:2001.08300 2020.
Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of gans for improved quality, stability, and variation. In International Conference on Learning Representations, 2018.
Durk P Kingma and Prafulla Dhariwal. Glow: Generative flow with invertible 1x1 convolutions. In Advances in Neural Information Processing Systems, volume 31, 2018.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to docu- ment recognition. Proceedings of the IEEE, 86(11):2278-233) 2324, 1998.
Jaewoo Lee, Minjung Kim, Yonghyun Jeong, and Youngmin Ro. Differentially private normalizing flows for synthetic tabular data generation. In AAAT Conference Artificial Intelligence, 2022.
David D Lewis, Yiming Yang, Tony Russell-Rose, and Fan Li. Revl: A new benchmark collection for text categorization rescarch. Journal of machine learning research, 5:361-397, 2004.
Ninghui Li, Min Lyu, Dong Su, and Weining Yang. Differential privacy: From theory to practice. Synthesis Lectures on Information Security, Privacy, & Trust, 8(4):1-138, 2016.
Seng Pei Liew, Tsubasa Takahashi, and Michihiko Ueno. PEARL: Data synthesis via private embeddings and adversarial reconstruction lcarning. In International Conference on Learning Representations, 2022.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), 2015.
14

---

Published in Transactions on Machine Learning Research (10/2023)
Yunhui Long, Boxin Wang, Zhuolin Yang, Bhavya Kailkhura, Aston Zhang, Carl A. Gunter, and Bo Li. G-PATE: Scalable differentially private data generator via private aggregation of teacher discriminators. In Advances in Neural Information Processing Systems, 2021.
Zclun Luo, Daniel Wu, Ehsan Adeli, and Li Fei-Fei. Scalable differential privacy with sparse network finetuning. In Proceedings of the IEEE/CVP Conference on Computer Vision and Pottern Recognition, pp. 5059-5068, 2021.
Ashwin Machanavajjhala, Daniel Kifer, Johannes Gehrke, and Muthuramakrishman Venkitasubramaniian. I-diversity: Privacy beyond k-anonymity, ACM Transactions on Knowledge Discovery from Data (TKDD), 111:3-css 2007.
Frank D McSherry. Privacy integrated queries: an extensible platform for preserig data analysis. In Proceedings of the 2009 ACM SIGMOD International Conference on Management of data, pp. 19-30, 2009.
Ily Mironov. Rényi differential privacy. In IEEE 30th Computer Security Foundations Sympostum (CSF),
pp. 263 275, 2017.
Arvind Narayanan and Vitaly Shmatikov. Robust de-anonymization of large sparse datasets. In IEEE Symposium on Security and Privacy, pp. 111 125, 2008.
Nicolas Papernot, Martin Abadi, Ulfar Erlingsson, lan Goodfellow, and Kunal Talwar. Semi-supervised knowledge transfer for deep learning from private training data. In International conference on learning representations, 2017.
Nicolas Papernot, Shuang Song, Nya Mironov, Ananth Raghunathan, Kunal Talwar, and Ulfar Frlingsson. Scalable private learning with PATE. International Conference on Learning Representations, 2018.
Bjarne Pfitzner and Bert Arnrich. Dpd-fvac: Synthetic data generation using federated variational autoen- coders with differentialll-privaae decoder. arXiv:2211.1159. 2022.
Alfréd Rényi. On measures of entropy and information. In Proceedings of the Fourth Berkeley Symposium on Mathemalicul Stalistics and Probability, volume 1, pp. 547 547-562, 562, 1961.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolttion image synthesis with latent diffusion models. In Proccedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10684 10695, 2022.
Pranav Subramani, Nicholas Vadivelu, and Gautam Kamath. Enabling fast diffcrentially private SGD via just-in-time compilation and vectorization. In NeurIPS, 2021.
Latanya Sweeney. k-anonymity: A model for protecting privacy. International Journal of Uncertainty, Fuzziness and Knowledge-Basse Systems, 10(05):557 570, 2002.
Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2818 2826, 2016.
Tsubasa Takahashi, Shun Takagi, Hajime Ono, and Tatsuya Komatsu. Differentially private variational autoencoders with lerm-wise gradient aggregation. arXiv:2006. 11204, 2020.
Reihanch Torkzadehmahani, Peter Kairouz, and Benedict Paten. DP-CGAN: Differentially private syn- thetic data and label generation. In IEEE/CVF Conference on Computer Vision and Pattern. Recognation Workshops (CVPRW), pp. 98-104, 2019.
Chris Waites and Rachel Cummings. Differentially private normalizing flows for privacy-preserinng density estimation. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, pp. 1000-1009, 2021.
15

---

Published in Transactions on Machine Learning Research (10/2023)
Boxin Wang, Fan Wu, Yunhui Long, Luka Rimanic, Ce Zhang, and Bo Li. Datalens: Scalable privacy pre- serving training via gradient compression and aggregation. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security, pp. 2146 2168, 2021.
Benjamin Weggenmann, Valentin Rublack, Michael Andrejczuk, Justus Mattern, and Florian Kerschbaum. Dp-vac: Human-readable text anonymization for online reviews with diffcrentially private variational auloencoders. In Proceedings of the ACM Web Conference 2022, pp. 721-731 731, 2022.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-MNIST. a novel image dataset for benchmarking machine learning algorithms. arXiv:1708.07747, 2017.
Liyang Xie, Kaixiang Lin, Shu Wang, Fei Wang, and Jiayu Zhou. Differentially privale generative adversarial network. arxiv:1802.06739, 2018.
Da Yu, Huishuai Zhang, Wei Chen, and Tie- Yan Liu. Do not let privacy overbill utility: Gradient embedding perturbation for private learning. In International Conference on Learning Representations, 2021.
16

---

Published in Transactions on Machine Learning Research (10/2023)
A Proof
Theorem 3.1. Let M, (i 11=1,22... 1,2,...,k) be k DP mechanisms, and each M, satisfies (e, D.DDP. Given a deterministic partitioning function f. let D1, De...., Dx be the disjoint partitions by executing f on D. satisfies (m...e(1.2....)) 6,, m)... 6))).... DDP
Releasing Mi(
Proof. Without loss of generality, given two neighboring datasets D and D', assume that D contains one more element than D'. Executing f on and D', we have partitions D1, D2..... Dj and D D/...... D': respectively. There exists such that (1) Dj contains one more element than Dj, and (2) D, D' for = 1,2,,,,... and * j. Denote Mi (Di).....MM(() by MMD). Since the subsets are disjoint from each other, rumning I algorithms on each subset is independent of each other. For any sequence t = (t1, .2..t.....tt. tx) of outputs of M,,.... Mr where 14 E (M)))(MM:), we have:
Pr[M(D) = 4 = Pr(MMi(D)) = 4 M2(222 = -2.A...AMM (a(a)) = [x]
(6)
= (D)( = =t,]
II =1.2...kk
Pr.M.(D.))= = t.l
(7)
(( =t)
II
Pr[M.(D')) =tel
(8)
sm 1,2
exp(G)
II Pr(M:((D)) = t.] +
II Pr(M,(D's) = t.
(9)
1112......k
= = -0+66 66,
II
Pr(Ma(D)) = (s))
(10)
2
= =t +
(11)
< exp( max 2
Pr)rr((D') = + max 6
(12)
2
B
Datasets
We briefly introduce the public datasets and associated preprocessing. Image size is shown in #channels x height x width. Images are normalized to the range of [0, 1].
MNIST (LeCun et al., 1998) & Fashion MNIST (Xiao et al. 2017): MNIST contains hand-written digits images, whereas Fashion MNIST contains cloth and shoe images. Images in both datasets are single- channel, in the size of 1 x 28 28, and have 10 classes. We adopt the official training and test split. 10k images from the training split are randomly held out the validation set.
CelebA (Liu et al. 2015): CelebA is dataset including face images of celebrities. Each image is in the size of 3 x 178 x 218 and has 40 binary attributes. All images are cropped to 3 x 178 x 178, and then resized to x 32 x 32. We also adopt the official training, validation and test split, but randomly select 50k images of each gender from the training split as our training set..
CelebA-HQ (Karras et al., 2018): CelebA-HQ is high-quality version of CclebA, which is commonly recognized as high-resolution image set. It consists of 30k images in total. We download a gender condi- tioned split (where images are resized to 3 x 256 256) by following this link 1999 images are randomly held out from the training split as the validation set.
C Rényi differential privacy (RDP)
Rényi differential privacy (RDP) extends ordinary DP using Rényi's a divergence (Rényi 1961 and provides tighter and easier composition property than the ordinary DP notion, thus we adopt RDP accumulate the privacy cost. Formally, recall:
17

---

Published in Transactions on Machine Learning Research (10/2023)
Table 5: Network configurations for different datasets in the experiments. #h conv denotes the number of hidden sizes in the convolutional layers. #h_ lin denotes the number of hidden sizes in the linear layers. #c denotes the length of latent code. #b means the number of blocks in flow.
Dataset
#h come encoder
#h come of decoder
#c
#b
#h lin of flow
MNIST
[32, 64]
[64, 32]
20
9
200
FMNIST
[32, 64
[64, 32]
20
9
200
CelebA
(64, 128, 256
[256, 128, 64]
32
200
Cel [16, 32, 64, 128, 256, 512]
[512, 256, 128, 64, 32, 16]
12
256
Definition C.1 ((a, RD- [Mironov] 2017 A randomised mechanism M is (a, c)-DDP for all adjacent
inputs D' Rényi's a-divergence (of order > 1) between the distribution of M(D) and M(D') satisfies:
DA (MM())))) == - log Ez-Q
PiZ)
Sc
(13)
Q12)
where and Q are the density of M(D) and M(D'), respectively (w.r.b. some dominating measure n.).
Importantly, mechanism satisfying (a,e)-RDP also satisfies(E+ 1/6 Conveniently, RDP is linearly composable:
8)-DP for any ò E (0,
Theorem C.1 (Sequential composition of RDP (Mironov 2017 If mechanism M, satisfies (a, RDD
for = 1,2,.... then releasing the composed mechanism .M.....,MMe Ma) satisfies (a, Σk r..-DD..
We also adopt the Gaussian mechanism for achieving RDP:
Let f D + Rp be Definition C.2 (Gaussian mechanism for RDP (Dwork et al. 2014 Mironov 2017 an arbitrary p-dimensional function with scnsitivity:
A max D.D"
(14)
for all adjacent datasets D, D' e D. The Gaussian mechanism Ma perturb the output of f with Gaussian noise:
M. = +D),
(15)
where I is identity matrix. Then, Ma satisfies (a, a(A2f)² 20 RD...
DP-SGD tracks the total privacy consumption as follows: (1) for each training itcration, compute the RDP privacy cost for subsampled batch where Gaussian mechanism is applied; (2) compose RDP mechanisms over training iterations; (3) convert RDP back to (c,8)-DP. The implementation details given below.
Framework
The schematic of DP-LFlow is shown in Figure 7
E Implementation
E.1 Flow models
Our code for flow generative models are adapted from public repos, i.c Glow and RealNVP, Hyperparamotors of the network are selected by comparing the performance on the validation set, and the selection results are given in Table
18

---

Published in Transactions on Machine Learning Research (10/2023)
private training
D1
Enc
Embedding
Dec
Embedding
Dec
(0,
Flow model
Flow model
D private datasçt (with & classes)
DR
Enc
Embedding
Dec
Embedding
Dec
2 ~ (0,
Flow model
Flow model
Training DP-LFlow
Published DP generator
Figure 7: The framework of DP-LFlow.
E.2 Privacy implementation
We use a public repo, i.e. pyvacy for implementing DP-SGD algorithm, as well as the total privacy calcu- lation. Pyvacy tracks the privacy loss by RDP accountant, which is a PyTTrrch implementation based on Tensorflow Privacy
For all datasots we use, we set subsampling rate as 0.1, training itcrations as 300, noise multiplier as 1.25 to target (10, 10- and 4.5 to target (1, 10-0--DDPP respectively. With better evaluation performance on the validation set, gradient clipping norms are set 0.1 for MNIST and Fashion MNIST, 0.01 for CelebA, and 10 for CelebA-HQ.
E.3
Fréchet Inception Distance (FID)
FID calculates the distance between the feature vectors extracted by InceptionV3 pool3 layer (Szegedy al. 2016 on real and synthetic samples. Specifically,
FID ||,, - pyll2 Tr(E, + Ey -(E,E,))
(16)
where X, ~ N(r, )x) and X, N ((g, 2n) are activations of InceptionV33 pool3 layer of real images and generated images, respectively, and Tr(A) refers to the trace a matrix A. Intuitively, a lower FID mcans the generation X, is more realistic (or more similar to X,). We use a Torch inplementation for computing FID, which will resize images and repeat channels three times for grayscale images to meet the input size requirement.
E.4
Classification task
We follow Cao al. (2021) for the classifier implementation. We import scikit-learn package for implemen- tation logistic regression classifier (e.g. from linearr.innerr...... waaammiiiiarmmdddlla model import LogisticRReression) with default parameter settings.
The MLP network consists of following layers: linear(inpul dim, 100) - ReLU - linear(100, output dim) + Softmax.
The CNN consists of following layers: Conv2d(input Convallinuttechanndds _chhnncl__n,,, 32, kernel kernel_size=3,, stride 2, padding-1)
Dro.out(p=O.5) ReLU + C32v2d(32, 64, kernel_size=3) size=3, stride 2, padding=1) Dropout(p=0.5) ReLU flatten linear(flatten dim, output. dim) Softmax.
19

---

Published in Transactions on Machine Learning Research (10/2023)
Both MLP and CNN are optimized by Adam with default parameters. All classifiers are trained on synthetic data, and we report test accuracy on real test data as the evaluation metric.
E.5 Baselines
All results of DP-MERF (Harder et al. 2021 are obtained by rumning their code with default parameters. It is worth mentioning that DP-MERF does not implement CelebA. We adapt their code CelebA by using the generative network they designed for SVHN with 16, 8, channels for three convolutional layers, respectively.
GS-WGAN only implements on (10, 10 D-DP. To target (1, DDD we tried two vanilla variations by tuning parameters of (10,10-9)-DD in their code i.c. increasing noise scale while keeping the rest param- eters unchanged, or decreasing the number of iterations while keeping the rest. parameters unchanged. We experimentally found that both variations will not generate meaningful images. The former variation is not even able to generate anything on Fashion MNIST, so instead present the latter variation for comparison.
All other results (e.g. mumbers the tables, generated images) are cited from papers as we specify.
20


Reviews:
[{"review_id": "XUD6cYvzlc", "reviewer": "Reviewer_CidX", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The authors have written this work very well, clearly elucidating their motivation, techniques and implementation details.\n2. From an empirical standpoint, this work outperforms previous works, indicating considerable progress.\n3. The concept, although simple, proves effective. This work is notably the first to achieve DP generation on high-resolution data.\n\nWeakness:\nMy major concern is that the method proposed in this work looks like a provisional solution to the challenge of DP to me. It does not genuinely address the problem in the practical application of DP, specifically the performance degradation of large network due to the added noise. It is clear now that large network can bring more benefits, we can also see it from the Figure 1 in this paper. While reducing the network size can enhance DP training efficiency, constrained network size also caps the performance that can be potentially obtained, which may prevent such frameworks from achieving satisfying results in the long run. Give this limitation, I question a little the future prospects of studying DP in this particular manner.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper explores the process of training flow-based generative models in the context of differential privacy. It is inspired by the recent success of diffusion model. Initially, this work applies an encoder to reduce the dimensionality of the input image and transforms the input image into some latent space. The generative model is then learned to generate latent codes that are distributed similar to those mapped from real data. According to the experimental results, the proposed method significantly outperforms the existing state-of-the-art DP generative models.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. On page 2: \"flow model usually apply BN model ... where the per-example gradient is not available\". Could you please provide more clarify on why per-example gradient is not available in this case. My understanding is that BN shares statistical information across examples, but it does not preclude the computation of the per-example gradient. \n2. The baseline flow models are proposed more than five years ago. Could you explain the choice of these specific models, why not choosing some more recent models? ", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "geTXygNibg", "reviewer": "Reviewer_2bxJ", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\nThe paper scales DP generative models to image datasets with reasonable DP parameters. Downstream training is quite accurate using this approach.\n\nThe approach permits density estimation, allowing for OOD detection.\n\nThe paper is clearly written.\n\nWeaknesses:\n\nThe contributions are limited by related work on DP diffusion models and Lee et al. which is cited. The work on DP diffusion models is also capable of generating reasonable images which lead to strong downstream performance [r1 r2].\n\nThe way the paper performs conditional generation is by using a new model for each partition of the dataset. This is a reasonable fix for the small datasets considered, but will run into challenges when training models on more diverse data, such as classifier or encoder-guided generation as is done in state of the art nonprivate image generation.\n\nOverall review:\n\nI recommend accepting the paper. The main issue with it is a lack of comparison to DP diffusion models.\n\nClaims and evidence: I think the claims made by the paper are mostly accurate, but the paper omits a comparison to recent work on DP diffusion models. I would like to see a comparison to this work.\n\nAudience: Many people in the community are interested in training differentially private generative models. I think this paper will be interesting to them.\n\n[r1] https://arxiv.org/abs/2210.09929\n\n[r2] https://arxiv.org/abs/2302.13861", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes a strategy called DP latent flows to do image generation, which trains a normalizing flow on a latent space, to generate images. This also allows the generative model to produce a density estimate, which can be used for tasks such as anomaly detection. They evaluate the performance of the approach on small image datasets, as well as CelebA.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Discuss work on DP diffusion models at least in the related work.", "broader_impact_concerns": "none", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "1il7H3580R", "reviewer": "Reviewer_8NY2", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The performance is state-of-the-art among the compared baselines.\n2. The proposed method is easy to understand and implement\n\nWeaknesses:\n1. The paper does seem lack novelty since it is just applying DP to latent flow models, and having one model for each class is intuitively straightforward and impractical when number of classes is large, or even infinite (e.g., text-guided image generation).\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes a new method for differentially private data generation, using normalizing flow. A few tricks are used to make DP training on normalizing flow work, one is to use latent flow, another is to replaced the conditional generation with different models for different classes. ", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "The key change in the paper seems to be using a separate model for each class, this would be impractical when number of classes is large, have the authors considered other alternatives? Adding an in depth discussion on this design choice would be highly appreciated.", "broader_impact_concerns": "No concern.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers were generally positive, and felt this was an important problem.
A main issue raised was about the lack of comparisons with recent works on DP diffusion models. The authors discussed this in their rebuttal, and promised to add some additional results and discussions contrasting their work with those on DP diffusion models.
Another point to add in the revision is the discussion with reviewer CidX about the claimed scaling laws.
These should be added for the final version.
Decision: Accept (poster)