Differentially Private Diffusion Models
Authors: Tim Dockhorn
Tianshi Cao
Arash Vahdat
Karsten Kreis
Venue: Accepted by TMLR ()

Abstract: While modern machine learning models rely on increasingly large training datasets, data is often limited in privacy-sensitive domains. Generative models trained with differential privacy (DP) on sensitive data can sidestep this challenge, providing access to synthetic data instead. We build on the recent success of diffusion models (DMs) and introduce Differentially Private Diffusion Models (DPDMs), which enforce privacy using differentially private stochastic gradient descent (DP-SGD). We investigate the DM parameterization and the sampling algorithm, which turn out to be crucial ingredients in DPDMs, and propose noise multiplicity, a powerful modification of DP-SGD tailored to the training of DMs. We validate our novel DPDMs on image generation benchmarks and achieve state-of-the-art performance in all experiments. Moreover, on standard benchmarks, classifiers trained on DPDM-generated synthetic data perform on par with task-specific DP-SGD-trained classifiers, which has not been demonstrated before for DP generative models. Project page and code: https://nv-tlabs.github.io/DPDM.

Published in Transactions on Machine Learning Research (08/2023)
Differentially Private Diffusion Models
Tim Dockhorn Stability AI*
timdockhorm@@gaalllccc
Tianshi Cao
tianshic@rvvidi..
NVIDIA
University of Toronto Vector Institute
Arash Vahdat. NVIDIA
avahdat@mvidiaa.cm
Karsten Kreis
kereis @nviddaa.ccoo com
NVIDIA
Reviewed on OpenReview: https: openreview. net/ net/ forum? id- JXP
Abstract
While modern machine learning models roly on increasingly large training datasets, data often limited in privacy-sensitive domains. Generative models trained with diffcrential privacy (DP) on sensitive data can sidestep this challenge, providing access to senthetic data instead. We build on the recent success of diffusion models (DMs) and introduce Differentially Private Diffusion Models (DPDMs), which enforce privacy using differentially private stochastic gradient descent (DP-SGD). We investigate the DM parameterization and the sampling algorithm, which turn out to be crucial ingredients in DPDMs, and propose noise multiplicity, a powerful modification of DP-SGD tailored to the training of DMs. We validate our novel DPDMs on image generation benchmarks and achieve state-of-the-ar performance in all experiments. Moreover, on standard benchmarks, classifiers trained on DPDM-generated synthetic data perform on par with task-specific DP-SGD-trained classifiers, which has not been demonstrated before for DP generative models. Project page and code: https: tttps://nv-taae github io/DPDM.
Introduction
Modern deep learning usually requires significant amounts of training data. However, sourcing large datasets in privacy-sensitive domains often difficult. To circumvent this challenge, generative models trained on sensitive data can provide access to large synthetic data instead, which can be used flexibly to train downstream models. Unfortunately, typical overparameterized neural networks have been shown to provide little to no privacy the data they have been trained For example, adversary may be able to recover training images of deep classifiers using gradients of the networks (Yin et al., 2021) reproduce training text sequences from large transformers (Carlini el al., 2021). Generative models may even overfit directly, generating data indistinguish- able from the data they have been trained on. In fact, overfitting and privacy-leakage of generative models are more relevant than ever, considering recent works that train powerful photo-realistic image generators on largo- scale Intereet-scraped data (Rombach et al., 2021; Ramesh et al., 2022; Saharia et al., 2022; Balaji et al., 2022).
To protect the privacy of training data, one may train their model using differential privacy (DP). DP is rigorous privacy framework that applies to statistical queries (Dwork et al., 2006; 2014). In case, this Work done during internship NVIDIA previous employment Stability AT.

---

Published in Transactions on Machine Learning Research (08/2023)
Sensttive Forward Pass
Calculation
Gradient
Objective:
1 = M(xe)l Do( oxk)
- x
Sensitive Backward Pass
Averaging over Nuiso Lovels
Averaging over
Σ 4x K
Noise Levels:
Private Backward
kE(K]
Objjeciive
Objective
Objective
Gradient Calculation:
ge = Val
Do Do S5x. ork.
Gradient Sanitization:
x clipe (Ko) + o.p
X2
X1
01
02
03
Diffusion Trajectories
Forward Diffusion Process X ((0,o
Figure 1: Information flow during training in our Differentially Private Diffusson Model (DPDM) for single training sample in green (i.e. batchsize B=1, another sample shown in blue). We rely on DP-SGD to guarantee privacy and use noise multiplicity; here, K=3. The diffusion is visualized for a one-dim. toy distribution (marginal probabilities in purple); our main experiments use high-dim. images. Note that for brevity in the visualization we dropped the index i, which indicates the minibatch element in Eqs. (6) and (7).
query corresponds to the training of a neural network using sensitive data. Differentially private stochastic gradient descent (DP-SGD) (Abadi et al., 2016) is the workhorse of DP training of neural networks. It preserves privacy by clipping and noising the parameter gradients during training. This leads to inevitable trade-off between privacy and utility; for instance, small clipping constants and large noise injection result in very private models that may be of little practical use.
DP-SGD has, for example, been employed to train generative adversarial networks (GANs) (Frigerio et al., 2019; Torkzadehmahani et al., 2019; Xie et al., 2018), which are particularly susceptible to privacy-leakage (Webster et. al., 2021). However, while GANs in the non-private setting can synthesize photo-realistic images (Brock et al., 2019; Karras et al., 2020b;a; 2021), their application in the private sctting is challenging. GANs are difficult to optimize (Arjovsky & Bottou, 2017; Mescheder et al., 2018) and prone mode collapse; both phenomena may be amplified during DP-SGD training.
Recently, Diffusion Models (DMs) have emerged as a powerful class of generative models (Song et al., 2021c; Ho et al., 2020; Sohl-Dickstein et al., 2015), demonstrating outstanding performance in image synthesis (Ho et al., 2021; Nichol & Dhariwal, 2021; Dhariwal & Nichol, 2021; Rombach et al., 2021; Ramesh et al., 2022; Saharia et al., 2022). In DMs, a diffusson process gradually perturbs the data towards random noise, while deep neural network learns denoise. DMs stand out not only by high synthesis quality, but also sample diversity, and a simple and robust training objective. This makes them arguably well suited for training under DP perturbations. Moreover, generation in DMs corresponds to iterative denoising process, breaking the difficult generation task into many small denoising steps that are individually simpler than the onc-shot synthesis lask performed by GANs and other traditional hodh. In particular, the denoising neural network that is learnt in DMs and applied repeatedly at each synthesis step less complex and smoother than the generator networks of onc-shot methods, as we validate in experiments on toy data (synthotically generated mixture of 2D Gaussians). Therefore, training of the denoising neural network is arguably less sensitive to gradient clipping and noise injection required for DP.
2

---

Published in Transactions on Machine Learning Research (08/2023)
Based on these observations, we propose Differentially Private Diffusion Models (DPDMs), DMs trained with rigorous DP guarantees based on DP-SGD. We thoroughly study the DM parameterization and sampling algorithm, and tailor them to the DP setting, We find that the stochasticity in DM sampling, which is empirically known to be crror-corecting (Karras et al., 2022), can be particularly helpful in DP-SGD training to obtain satisfactory perceptual quality. We also propose noise multiphicity, where a single training data sample is re-used for training at multiple perturbation levels along the diffusion process (sce Fig. 1). This simple yet powerful modification of the DM training objective improves learning at no additional privacy cost. We validate DPDMs on standard DP image generation tasks, and achieve state-of- the-arrt performance by large margins, both in terms of perceptual quality and performance of downstream classifiers trained on synthetically generated data from our models. For example, on MNIST we improve the seate-of-the-arr FID from 56.2 to 23.4 and downstream classification accuracy from 81.5% to 95.3% for the privacy setting DP-(s=1, 0=10-5). We also find that classifiers trained on DPDM- synthetic data perform on par with task-specific DP-classifiers trained on rcal data, which has not been demonstrated before for DP generative models.
We make the following contributions: (i) We carefully motivate training DMs with DP-SGD and introduce DPDMs, the first DMs trained under DP guarantees. (ii) We study DPDM parameterization, training setting and sampling in detail, and optimize it for the DP sctup. (iii) We propose noise multiplicity to efficiently boost DPDM performance. (iv) Experimentally, we significantly surpass the state-of-the-art the-art in DP synthesis on widely-studied image modeling benchmarks. (v) We demonstrate for the first time that classifiers trained on DPDM-generated data perform on par with task-specific DP-trained discriminative models. This implies a very high utility of the synthetic data generated by DPDMs, delivering on the promise of DP generative models as an effective data sharing medium. Finally, we hope that our work has implications for the literature DMs, which now routinely trained on ultra large-scale datasets of diverse origins.
2 Background
2.1
Diffusion Models
We consider contimuuus-time DMs (Song et al., 2021c) and follow the presentation of Karras et al. (2022).
Let (x) denote the data distribution and p(x; the distribution obtained by adding i.i.d. o--variacce Gaussian noise to the data distribution. For sufficiently large maax, p(x; max) is almost indistinguishable from miax- variance Gaussian noise. Capitalizing on this observation, DMs sample (high variance) Gaussian noise Xg N (0, m and sequentially denoise Xg into x; ~ p(x;; oi), p(), i E [O, with o, < 0,-1 (oo- = mmax) ). If MM-- 0, then Xy is distributed according to the data.
Sampling. In practice, the sequential denoising is often implemented through the simulation of the Probobility Flow ordinary differential equation (ODE) (Song et al., 2021c)
dx = loggxx dt,
(1)
where Vx logp(x;o) o) is the score function (Hyvärinen, 2005). The schedule att) : [0, 1] + R., user-specified and att) denotes the time derivative of (t).. Alternatively, we may also sample from a stochastic differential equation (SDF) (Song et al., 2021c; Karras et al., 2022):
dx= = V
di B(t)) logp(x; (()) dt + (2)
(2)
Probability Flow ODE; see Eq. (1)
Langevin diffusion component
where dws; is the standard Wiener process. In principle, given initial samples xo N'' (0, oaax), simulating either Probability Flow ODE or SDE produces samples from the same distribution. In practice, though, neither ODE nor SDE can be simulated exactly: Firstly, any numerical solver inevitably introduces discretization errors. Secondly, the score function is only accessible through a model (x(;;)) that needs to be learned; replacing the score function with an imperfect model also introduces an error. Empirically, the ODE formulation has been used frequently to develop fast solvers (Song et al., 2021a; Zhang & Chen, 2022; Lu et al., 2022; Liu et al., 2022; Dockhorn et al., 2022a), whereas the SDE formulation often leads to higher quality samples (while requiring more steps) (Karras et al., 2022). One possible explanation for the latter observation is that the Langevin diffusion component in the SDE at any time during the synthesis process actively drives the
3

---

Published in Transactions on Machine Learning Research (08/2023)
process towards the desired marginal distribution p(x; d), whereas errors accumulate in the ODE formulation, even when using many synthesis steps. In fact, it has been shown that as the score model 8o improves, the performance boost that can be obtained by an SDE solver diminishes (Karras et al., 2022). Finally, note that we are using classificr-free guidance (Ho & Salimans, 2021) to perform class-conditional sampling in this work. For details on classifier-frew guidance and the numerical solvers for Eq. (1) and Eq. (2), we refer to App. C.3.
Training. DM training reduces to learning the score model sg. The model can, for example, be parameterized 8p (D)(x; x)/o² (Karras et al., 2022), where Dp is learnable denoiser that,
Vx loggp(x; given noisy data point x + n, x Pdata (x), ~ N (0, and conditioned on the noise level 0, tries to predict the clean x. The denoiser Do can be trained by minimizing an Lz-loss
Ex Ex (xxpaau(()))
Ao + -)) -x
(3)
where p(a,n) p(o) W W 0n²)
and Ao: R+ - R+ is weighting function. Previous works proposed various denoiser models Do, noise distributions p(a), and weightings Ao We refer to the triplet (Do, p. A) as the DM config. Here, we consider four such configs: variance preserving (VP) (Song et al., 2021c), variance exploding (VE) (Song et al., 2021c), v-prediction (Salimans & Ho, 2022), and EDM Karras et al. (2022); App. C.I for details.
2.2 Differential Privacy
DP is a rigorous mathematical dcfinition of privacy applicd to statistical queries; in our work the queries correspond to the training of a neural network using sensitive training data. Informally, training is said to be DP, if, given the trained weights 0 of the network, an adversary cannot tell with certainty whether a particular data point was part of the training data. This degree of certainty is controlled by two positive parameters E and 6: training becomes more private as E and decrease. Note, however, that there is an inherent trade-off bet utility and privacy: very private models may be of little to practical use. To guarantee sufficient amount of privacy, rule of thumb, ō should not be larger than I/N, where N is number of training points {x:], and should be a small constant. More formally, we refer to (e, 6)-DP defined as follows (Dwork et al., 2006):
- R. with domain D and range R satisfies (s, 8)-DP if for any two datasets d, d' E DD differing by at most one entry, and for any subset of outputs S _ R it holds that
Definition 2.1. (Differential Privacy) A randomized mechanism M: D
Pr(M(d) E ES Pr E +
(4)
DP-SGD. We require a DP algorithm that trains a neural network using sensitive data. The workhorse for this particular task is differentially private stochastic gradient descent (DP-SGD) (Abadi et al., 2016). DP-SGD is a modification of SGD for which por-sample-gradione are clipped and noise is added to the clipped gradients; the DP-SGD parameter updates are defined as follows
0-0- B iE clipe (Vol.00) + Cz).
17
E
(5)
where (O op B is a B-sized subset of N]. drawn uniformly at random, 4 is the loss function for data point x;, 17 is the learning rate, and the clipping function is clipo min {1, C/Ilglla) DP-SGD can be adapted to other first-order optimizers, such as Adam (McMahan et al., 2018).
Privacy Accounting. According to the Gaussian mechanism (Dwork et al., 2014), a single DP-SGD update Privacy accounting methods can be used to compose the privacy cost of multiple DP-SGD training updates and to determine the variance oh needed to satisfy (e, 8D for a particular number of DP-SGD updates with clipping constant C and subsampling rate B/N. Also see App. A.
(Eq. (5)) satisfies (c, 8)DD if of > 2 (ogg(125/0)CC)
3 Differentially Private Diffusion Models
We propose DPDMs, DMs trained with rigorous DP guarantees based on DP-SGD (Abadi et al., 2016). DP- SGD is well-established method to train DP neural networks and our intention is not to re-invent DP-SGD;
4

---

better than GANs' (Dhariwal Published in Transactions on Machine Learning Research (08/2023)
1000
100
10
(F(a)
1
0.1
Denoiser ((;oo)
0.01
DM
0.001
GAN
0.005
0.02
0.1
0.5
1
2
5
Noise level
Figure 2: Frobenius norm of the Jacobian Jr(a) of the denoiser D(,, о) and constant Frobenius norms of the Jacobians JF of the sampling functions defined by the DM and a GAN. App. for experiment details.
instead, the novelty in this work lies in the combination of DMs with DP-SGD, modifications of DP-SGD specifically tailored to the DP training of DMs, as well as the study of design choices and training recipes that greatly influence the performance DPDMs. Combinations of DP-SGD with GANs have been widely studied (Frigerio et al., 2019; Torkzadehmahani et al., 2019; Xie et al., 2018; Bie et al., 2022), motivating a similar line of research for DMs. To the best of our knowledge, we are the first to explore DP training of DMs. In Sec. 3.1, we discuss the motivation for using DMs for DP generative modeling. In Sec. 3.2, then discuss training and methodological details as well as DM design choices, and we prove that DPDMs satisfy DP.
3.1 Motivation
(i) Objective function. GANs have so far been the workhorse of DP generative modeling (sce Sec. 4), even though they are generally difficult to optimize (Arjovsky & Bottou, 2017; Mescheder et al., 2018) due to their adversarial training and propensity to mode collapse. Both phenomena may be amplified during DP-SGD training. DMs, in contrast, have been shown to produce outputs as good & Nichol, 2021), while being trained with very simple regression-like L2-loss (Eq. (3)), which makes them robust and scalable in practice. DMs are therefore arguably also well-suited for DP-SGD-based training and offer better stability under gradient clipping and noising than adversarial training frameworks.
(ii) Sequential denoising. In GANs and most other traditional generative modeling approaches, the generator directly learns the sampling function, iee. the mapping of latents to synthesized samples, end-to-end. In contrast, the sampling function in DMs defined through a sequential denoising process, breaking the difficult generation task into many small denoising steps which are individually less complex than the ono-shot synthesis task performed by, for instance, a GAN generator. The denoiser neural network, the learnable component in DMs that is evaluated once per denoising step, is therefore simpler and seoother than the one- shot generator networks of other methods. We fit both a DM and GAN to a two-dimensional toy distribution (mixture of Gaussians, see App. E) and empirically verify that the denoiser is indeed significantly less complex (quantified by the Frobenius norm of the Jacobian) than the generator learnt by the GAN and also than the end- to-end multi-step synthesis process (Probability Flow ODE) of the DM (sce Fig. 2; we calculate denoiser Jr(a) at varying noise levels a). Generally, more complex functions require larger neural networks and are more difficult to learn. In DP-SGD training we only have a limited number of training iterations available until the privacy budget is depleted. Consequently, the fact that DMs require less complexity out of their neural networks than typical one-shot generation methods, while still being able to represent expressive generative models due to the iterative synthesis process, makes them likely well-suited for DP generative modeling with DP-SGD.
(iii) Stochastic diffusion model sampling. As discussed in Sec. 2.1, generating samples from DMs with stochastic sampling can perform better than deterministic sampling when the score model not learned well. Since we replace gradient estimates in DP-SGD training with biased large variance estimators, we
5

---

Published in Transactions on Machine Learning Research (08/2023)
cannot expect perfectly accurate score model. In Sec. 5.2, we empirically show that stochastic sampling can in fact boost perceptual synthesis quality in DPDMs as measured by FID.
3.2 Training Details, Design Choices, Privacy
The clipping and noising of the gradient estimates DP-SGD (Eq. (5)) pose a major challenge for efficient op- timization. Blindly reducing the added noise could be fatal, it decreases the number of training iterations al- lowed within a certain (e, 6)-DP budget. Furthermore, as discussed the Lg-norm of the noise added in DP-SGD scales linearly to the number of parameters. Consequently, settings that work well for non non-private DMs, such relatively small batch sizes, large number of training iterations, and heavily overparameterized models, may not work well for DPDMs. Below, we discuss how we propose to adjust DPDMs for successful DP-SGD training,
Noise multiplicity. Recall that the DM objective in Eq. (3) involves three expectations. As usual, the expectation with respect to the data distribution Pdata (x) is approximated using mini-batching. For non- private DMs, the expectations over and n are generally approximated using a single Monte Carlo sample (o,, n,) ~ p(o) (0,o²) per data point x;, resulting in the loss for training sample
4 = D(xx) + +,; - x:112
(6)
The estimator ( is very noisy in practice. Non-private DMs counteract this by training for a large number of iter- ations in combination with an exponential moving average (EMA) the trainable parameters 0 (Song & Ermon, 2020). When training DMs with DP-SGD, we incur a privacy cost for each iteration, and therefore prefer a small number of itcrations. Furthermore, since the per-example gradient clipping as well as the noise injection induce additionn ional variance, we would like our objective function to be less noisy than in the non-DP case. We achieve this by estimating the expectation over and using an average over K noise samples, {(osk, nx)) P(o) (o, o²) for cach data point x,, replacing the non-private DM objective 4, in Eq. (6) with
Σ K 1, = K
k=1 M(ow))
+ D.k. o.h) -x,12-
(7)
Importantly, we show that this modification comes at no additional privacy cost (also see App. A). We call this simple yet powerful modification of the DM objective, which is tailored to the DP setup, noise multiplicity. Theorem 1. The variance of the DM objective (Eq. (7)) decreases with increased noise multiplicity K 1/K.
Proof in App. D. Intuitively, the key is that we first create relatively accurate low-variance gradient estimate by averaging over multiple noise samples before performing gradient sanitization in the backward pass via clipping and noising. This averaging process increases computational cost, but provides better utility at the same privacy budget, which is the main bottleneck in DP generative modeling; see App. D.3 for further discussion. We empirically showease in Sec. 5.2 that the variance reduction induced by noise multiplicity is key factor in training strong DPDMs. In Fig. 3, we show that the reduction of variance in the DM objective also empirically leads to lower variance gradient estimates (sce App. D for experiment details). The noise multiplicity mechanism is also highlighted in Fig. 1: the figure describes the information flow during training for single training sample (i.e., batch size B 1). Note that noise multiplicity is loosely inspired by augmentation multiplicity (De et al., 2022), a technique where multiple augmentations per image are used to train classifiers with DP-SGD. In contrast to augmentation multiplicity, our novel noise multiplicity is carefully designed specifically for DPDMs and comes with theoretical proofs on its variance reduction. The reader may find a more detailed discussion on the difference between noise multiplicity and data multiplicity (for DPDMs) in App. D.4.
Neural networks sizes. Current DMs are heavily overparameterized! For example, the current state-of-the- art image generation model (in terms of perceptual quality) on CIFAR-10 uses more than 100M parameters, despite the dataset consisting of only 50k training points (Karras et al., 2022). The per-cxample clipping operation of DP-SGD requires the computation of the loss gradient on each training example Val,, rather than the minibatch gradient. In theory, this increases the memory footprint by at least O(B); however, in common DP frameworks, such as Opacus (Yousefpour et al., 2021), which we use, the peak memory requirement is O(B²) compared to non- ooivviiv training (recent methods such as ghost clipping (Bu et al., 2022) require less memory, but are not widely implemented) On top of that, DP-SGD generally already relies on significantly
6

---

Published in Transactions on Machine Learning Research (08/2023)
x10
3.0
x10
3-
Average gradient variance
2.5
==8
K 32
Count
2-
2.0
1
1.5
0
1.0
10-11
10-3
10-5
10-2
Gradient variance
Noise multiplicity
32
Figure Increasing noise multiplicity leads to significant variance reduction of parameter gradient estimates during training (note logarithmic axis in inset). Enlarged version in Fig. 6.
increased batch size, when compared to non-private training, to improve the privacy-utility trade-off. As a result, we train very small neural networks for DPDMs, when compared to their non-DP counterparts: our models on MNIST SAAIST/Faahion-MNISSS MNIST and CelobA have 1.75M and 1.80M paramoters, respectively. Furthermore, we found smaller models to perform better across our experiments which may be due to the Lg-norm of the noise added in DP-SGD update scaling linearly with the number of parameters. This is in contrast to recent works in supervised DP learning, which show that larger models may perform better than smaller models (De et al., 2022; Li et al., 2022b; Anil et al., 2021; Li et al., 2022a).
Diffusion model config. In addition to network size, we found the choice of DM config, ie., denoiser parameterization Dg, weighting function X(o), and noise distribution p(o), to be important. In particular the latter is crucial obtain strong results with DPDMs. In Fig. 4, visualize the noise distributions of the four configs under consideration. We follow Karras et al. (2022) and plot the distribution p(log o) over the log-noise level. Especially for high privacy settings (small E), we found it important to use distributions that give sufficiently much weight to larger o, such as the distribution of v-prediction (Salimans & Ho, 2022). It. known that at large the DM learns the global, coarse structure of the data, i.e., the low frequency content in the data (images, in our case). Learning global structure reasonably well is crucial to form visually coherent images that can also be used to train downstream models. This is relatively easy to achieve in the non-DP setting, due to the heavily smoothed diffused distribution at these high noise level. At high privacy levels, however, even training at such high noise levels can be challenging due to DP-SGD's gradient clipping and noising. We hypothesize that this why beneficial to give relatively more weight to high noise levels when training in the DP setting. In See. 5.2, we empirically demonstrate the importance of the right choice of the DM config.
0.4
EDM v-prediction
VE
VP
0.3
p(loga)
0.2
0.1
0.0
0.002
0.02
0.1
1
10
80
Noise level
Figure 4: Noise level sampling for differ- ent DM configs; see App. C.I.
DP-SGD settings. Following De et al. (2022) we use very large batch sizes: 4096 and 2048 on CelobA. Similar to previous works (De et al., 2022; Kurakin et al., 2022; Li et al., 2022b), we found that small clipping constants C work better than larger clipping norms; in particular, we found C = to work well across our experiments. Decreasing C' even further had little effect; in contrast, increasing C' significantly worsened performance. Similar non-private DMs, we use an EMA of the learnable parameters 0. Incidentally, this has recently been reported to also have positive effect on DP-SGD training of classifiers by De et. al. (2022).
MNIST
7

---

Published in Transactions on Machine Learning Research (08/2023)
Algorithm DPDM Training
Privale data set = (xy)}=1, subsampling rate B/N, DP noise scale oDp, clipping constant C, sampling function Poisson Sample (Alg. 2), denoiser Do with initial parameters 0, noise distribution p(a), learning rate ? total steps 7, noise multiplicity K Adam (Kingma & Ba, 2015) optimizer
Input:
Output: Trained parameters 0
for t to do
B ~ Poisson Sample(N, B/N) for foriE E do
(itk) 1== ((,) o²)
R Ei=l Moraa) Po(xxttttxss Tih) xxll
end for
Ghatch = B Eies clipe (Vol.)
Chetch = Gbatch + (C/B)z, N/
A 0= 0 - Adam (dam(Gaatoo))
end for
Privacy. We formulate privacy protection under the Rényi Differential Privacy (RDP) (Mironov, 2017) framework (see Definition A.1), which can be converted to (c, DP. For an algorithm for DPDM training with noise multiplicity see Alg. 1. For the sake of completeness we also formally prove the DP of DPDMs (DP of relcasing sanitized training gradients Gbatchi):
Theorem 2. For noise magnitude ODP, releasing Ghatch in Alg. 1 satisfies (2,
The proof can be found in App. A. Note that the strength of DP protection is independent of the noise multiplicity, as discussed above. In practice, we construct mini-batches by Poisson Sampling (See Alg. 2) the training dataset for privacy amplification via sub-sampling (Mironov et al., 2019), and compute the overall privacy cost of training DPDM via RDP composition (Mironov, 2017). Tighter privacy bounds, such as the one developed in Gopi et al. (2021), may lead to better results but are not widely implemented (not in Opacus (Yousefpour et al., 2021), the DP-SGD library we use).
4 Related Work
In the DP generative learning literature, several works (Xie et al., 2018; Frigerio et al., 2019; Torkzadchmahani et al., 2019; Chen et al., 2020) have explored applying DP-SGD (Abadi et al., 2016) to GANs, while others (Yoon et al., 2019; Long et al., 2019; Wang et al., 2021) train GANs under the PATE (Papernot et al., 2018) framework, which distills private teacher models (discriminators) into public student (generator) model. Apart from GANs, Acs et al. (2018) train variational autoencoders on DP-sanitized data clusters, and Cao et al. (2021) use the Sinkhorn divergence and DP-SGD.
DP-MERF (Harder et al., 2021) was the first work to perform one-shot privatization on the data, followed by non-private learning. Tt uses differentially private random Fourier features to construct Maximum Mean Discrepancy loss, which is then minimized by a generative model. PEARL (Liew al., 2022) instead minimizes an empirical characteristic function, also based on Fourier features. DP-MEPF (Harder et al., 2022) extends DP-MERF to the mixed public-private setting with pre-trained feature extractors. While these approaches are efficient in the high-privacy/sman /small dataset regime, they are limited in expressivity by the data statistics that can be extracted during one-shot privatization. As a result, the performance of these methods does not scale well in the low-privacy/ cairaccyllrrg /large dataset regime.
In our experimental comparisons, we excluded Takagi al. (2021) and Chen et al. (2022) duc concerns regarding their privacy guarantees. The privacy analysis of Takagi al. (2021) relies on the Wishart mechanism, which has been retracted due to privacy leakage (Sarwate, 2017). Chen et al. (2022) attempt to train score-based model while guarantecing differential privacy through data-dependent randomized response mechanism. In App. B, we prove why their proposed mechanism leaks privacy, and further discuss other sources of privacy leakage.
Our DPDM relics on DP-SGD (Abadi et al., 2016) enforce DP guarantecs. DP-SGD has also been used train DP classifiers (Dörmann et al., 2021; Tramer & Bonch, 2021; Kurakin et al., 2022). Recently, De et al. (2022) demonstrated how to train very large discrininative models with DP-SGD and proposed augmentation
8

---

Published in Transactions on Machine Learning Research (08/2023)
Table 1: coass-coniiional DP image generation performance (MNIST & Fashion-MNISTT), For PEARL (Liew et al., 2022), we train models and compute metrics ourselves (App. F.1). All other results taken from the literature. DP-MEPF (1) uses additional public data for training (only included for completeness).
MNIST
Pashion-MNIST
Method
DP-s
Ace (%)
Acc (%6
FID
FID
Log Reg
ML.
CNN
Log Reg
ML.
CNN
DPDM (FID) (ours)
0.2
61.9
65.3
65.8
71.9
78.4
53.6
55.3
57.0
DPDM (Acc) (ours)
0.2
104
81.0
81.7
86.3
128
70.4
71.3
72.3
PEARL (Liew et al., 2022)
133
76.2
77.1
77.6
160
70.0
70.8
68.0
DPDM (FID) (ours)
1
23.4
83.8
87.0
93.4
37.8
71.5
71.7
73.6
DPDM (Aec) (ours)
1
35.5
86.7
91.6
95.3
51.4
76.3
76.9
79.4
PEARL (Liew et al., 2022)
1
121
76.0
79.6
78.2
109
74.4
74.0
68.3
DPGANr (Bie et al., 2022)
1
56.2
80.1
1218
68.0
DP-HP (Vinaroz al., 2022)
81.5
72.3
DPDM (FID) (ours)
5.01
90.5
94.6
97.3
18.6
S0.4
81.1
84.9
DPDM (Aec) (ours)
10 6.65
90.8
94.8
98.1
19.1
81.1
83.0
86.2
PEARL (Liew et al., 2022)
10
116
76.5
78.3
78.8
102
72.6
73.2
64.9
DPGANr (Bie et al., 2022)
10
13.0
95.0
56.8
74.8
DP-Sinkboru (Cao et al., 2021)
10
48.4
82.8
82.7
83.2
128.3
75.1
74.6
71.1
G-PATE (Long et ul., 2019)
10
150.62
80.92
171.90
69.34
DP-CGAN (Torkzadehmmaani et al., 2019)
10
179.2
60
60
63
243.8
51
50
46
DataLens (Wang et al., 2021)
173.5
80.66
167.7
70.61
DP-MERF (Harder al., 2021)
116.3
79.4
78.3
82.1
132.6
75.5
74.5
75.4
GS-WGAN (Chen et al,, 2020)
61.3
79
79
131.3
65
65
DP-MEPF (61) (Harder et al., 2022) (1)
0.2
72.1
69.0
DP-MEPF (2))02) (Harder et al., 2022) (1)
0.2
75.8
79.9
72.5
70.4
DP-MEPF (61) (Harder et al., 2022) (1)
1
79.0
87.5
76.2
75.0
DP-MEPF (2) (Harder et al., 2022) (f)
1
82.5
89.3
75.4
74.7
DP-MEPF (61) (Harder al., 2022) (t)
10
80.8
88.8
75.5
75.5
DP-MEPF (61, du) (Harder et al., 2022) (I)
83.4
89.8
75.7
76.0
multiplicity, which related our noise multiplicity, as discussed in Sec. 3.2. Furthrmore, DP-SGD has been utilized to train and fine-tune large language models (Anil et al., 2021; Li et al., 2022b; Yu et al., 2022), to protect sensitive training data in the medical domain (Ziller et al., 2021a;b; Balelli et al., 2022), and to obscure geo-spatial location information (Zeighami et al., 2022).
Our work builds on DMs and score-based generative models (Sohl-Dickstein et al., 2015; Song et al., 2021c; Ho et al., 2020). DMs have been used prominently for image synthesis (Ho et al., 2021; Nichol Dhariwal, 2021; Dhariwal & Nichol, 2021; Rombach et al., 2021; Ramesh et al., 2022; Saharia et al., 2022; Balaji et al., 2022) and other image modeling tasks (Meng et al., 2021; Saharia et al., 2021a;b; Li et al., 2021; Sasaki et al., 2021; Kawar et al., 2022). They have also found applications in other areas, for instance in audio and speech generation (Chen et al., 2021; Kong et al., 2021; Jeong et al., 2021), video generation (Ho et al., 2022b;a; Singer et al., 2023; Blattmann et al., 2023) and 3D synthesis (Luo & Hu, 2021; Zhou et al., 2021; Zeng et al., 2022; Kim et al., 2023). Methodologically, DMs have been adapted, for example, for fast sampling (JoliceerrMaartines et al., 2021; Song et al., 2021a; Salimans & Ho, 2022; Dockhorn et al., 2022b; Xiao et al., 2022; Watson et al., 2022; Dockhorn et al., 2022a) and maximum likelihood training (Song et al., 2021b; Kingma et al., 2021; Vahdat et al., 2021). To the best of our knowledge, we are the first to train DMs under differential privacy guarantees.
5 Experiments
In this section, we present results of DPDM on standard image synthesis benchmarks. Importantly, note that all models are private by construction through training with DP-SGD. The privacy guarantee is given by the (e,8) parameters of DP-SGD, clearly stated for each experiment below.
Datasets. We focus on image synthesis and use MNIST (LeCun et al., 2010), Fashion-MNIS' (Xiao et al., 2017) (28x28), and CelebA (Tiu et al., 2015) (downsampled to 32x32). These datasets standard

---

Published in Transactions on Machine Learning Research (08/2023)
Figure 5: Fashion-MNIST images generated by, from top to bottom, DP-CGAN (Torkvadehmahani et al., 2019), DP-MERF (Harder et al., 2022), Datalens (Wang et al., 2021), G-PATE (Long et al., 2019), GS- (Chen ot al., 2020), DP-Sinkhorn (Cao et al., 2021), PEARL (Liew et al., 2022), DPGANr (Bic et al., 2022) (all above bar), and our DPDM (below bar) using the privacy budget 1=10. See App. F.5 for more samples.
benchmarks in the DP generative modeling literature. In App. G, we consider more challenging datasets and provide initial results.
Architectures. We implement the neural networks of DPDMs using the DDPM+ architecture (Song et al., 2021c). See App. C.2 for details.
Evaluation. We measure sample quality via Fréchet Inception Distance (FID) (Heusel et al., 2017). On MNIST and -ashion-MNISTT we also assess utility of class-labeled generated data by training classifiers on synthesized samples and compute class prediction accuracy on real data. As is standard practice we consider logistic regression (Log Reg), MLP, and CNN classifiers; see App. F.1 for details.
Sampling. We sample from DPDM using (stochastic) DDIM (Song et al., 2021c) and the Churn sampler introduced in (Karras al., 2022). See App. C.3 for details.
Privacy implementation: We implement DPDMs in PyTorch (Paszke et al., 2019) and use Opacus (Yousef- pour et al., 2021), DP-SGD library in PyTorch, for training and privacy accounting. We use 5=10-5 for MNIST and Mashion-MNIST, and 8-10-6 for CelebA. These values are standard (Cao et al., 2021) and chosen such that smaller than the reciprocal of the number training images. Similar to existing DP generative modeling work, we do not account for the (small) privacy cost of hyperparameter tuning. However, training and sampling very robust with regards to hyperparameters, which makes DPDMs an ideal candidate for real privacy-critical situations; see App. C.4.
5.1 Main Results
Class-conditional gray scale image gencration. For MNIST and Fashion-MNIST, we train models for three privacy settings: 8- 1, 10) (Tab. 1). Informally, the three settings provide high, moderate, and low amounts of privacy, respectively. The DPDMs use the v-prediction DM config (Salimans & Ho, 2022) for e=0.2 and the EDM config (Karras al., 2022) for c=(1, 10); see Sec. 5.2. We use the Churn sampler (Karras et al., 2022): the two settings (FID) and (Ace) are based on the same DM, differing only in sampler setting; see Tab. 14 and Tab. 15 for all sampler settings.
10

---

Published in Transactions on Machine Learning Research (08/2023)
Table 2: Class prediction accuracy on real test data. DP-SGD: Classifiers trained directly with DP-SGD and real training data. DPDM: Classifiers trained non-privately on synthesized data from Dr-SGD-trained DPDMs (using 60,000 samples, following Cao al. (2021)).
MNIST
Pashion-MNIST
DP-s
Log Reg
MLP
CNN
Log Reg
MLP
CNN
DP-SGD
DPDM
DP-SGD
DPDM
DP-SGD
DPDM
DP-SGD
DPDM
DP-SGD
DPDM
DP-SGD
DPDM
0.2
83.8
81.0
82.0
81.7
69.9
86.3
74.8
70.4
73.9
71.3
59.5
72.3
1
89.1
86.7
89.6
91.6
88,2
95.3
79.6
76.3
79.6
76.9
70.5
79.4
10
91.6
90.8
92.9
94.8
96.4
98.1
83.3
81.1
83.9
83.0
77.1
86.2
Table 3: DM config ablation on MNIST for ==0.2. See Tab. 12 for extended results.
DM config
FID
CNN-Acc (%)
VP (Song et al., 2021c)
197
24.2
(Song et al., 2021c)
171
13.9
v-prediction (Salimans Ho, 2022)
97.8
84.4
EDM (Karras et al., 2022)
119
49.2
DPDMs outperform all other existing models for all privacy settings and all metrics by large margins (see Tab. 1). Interestingly, DPDM also outperforms DP-MEPF (Harder et al., 2022), method which is trained on additional public data, in 22 out of 24 setups. Generated samples for ==10 are shown in Fig. 5. Visually, DPDM's samples appear to significantly higher quality than the baselines'
Comparison to DP-SGD-trained classifiers. Is it better train task-specific private classifier with DP-SGD directly,
non-private classifier trained on DPDM's synthethized data perform well downstream tasks? To answer this question, we train private classifiers with DP-SGD real (training) data and compare them to our classifiers learnt using DPDM-syuthesized data (details in App. F.3). For fair comparison, we are using the same architectures that we have already been using in our main experiments to quantify downstream classification accuracy (results in Tab. 2; we test on real (test) data). While direct DP-SGD training on real data outperforms the DPDM downstream classifier for logistic regression in all six setups (in line with empirical findings that is casier train classifiers with few parameters than large ones with DP-SGD (Tramer & Boneh, 2021)), CNN classifiers trained on DPDM's synthetic data generally outperform DP-SGD-trained classifiers. These results imply very high utility of the synthetic data generated by DPDMs, demonstrating that DPDMs can polentially be used as an elfective, privacy-preseringg data sharing medium in practice. In fact, this approach is beneficial over training task-specific models with DP-SGD, bocause a user can generate as much data from DPDMs as they desire for various downstream applications without further privacy implications. To the best of our knowledge, it has not been demonstrated before in the DP generative modeling literature that image data generated by DP generative models can be used to train discriminative models on-par with directly DP-SGDD-rained task-specific models.
Table 4: Unconditional CelebA generative performance. G-PATE and DataLens (t) use ô 10-5 (less privacy) and model images at 64x64 resolution.
Method
DP-g
FID
DPDM (ours)
1
71.8
DPDM (ours)
10
21.1
DP-Sinkhorn (Cao et al., 2021)
10
189.5
DP-MERF (Harder al., 2021)
10
274.0
G-PATE (Long al., 2019) (f)
10
305.92
DataLens (Wang et. al., 2021) (t)
10
320.8
11

---

Published in Transactions on Machine Learning Research (08/2023)
Table Sampler comparison on MNIST (see Tab. 13 for results on Fashion-MLNIST). We compare the Churn sampler (Karras et al., 2022) to DDIM (Song et al., 2021a).
Acc (%)
Sampler
DP-s
FID
Log Reg
MLP
CNN
Churn (FID)
0.2
61.9
65.3
65.8
71.9
Churn (Ace)
0.2
104
81.0
81.7
86.3
Stochastic DDIM
0.2
97.8
80.2
81.3
84.4
Deterministic DDIM
0.2
120
81.3
82.1
84.8
Churn (FII)
23.4
83.8
87.0
93.4
Churn (Ace)
35.5
86.7
91.6
95.3
Stochastic DDIM
34.2
86.2
90.1
94.9
Deterministic DDIM
50.4
85.7
91.8
94.9
Churn (FID)
10
5.01
90.5
94.6
97.3
Churn (Ace)
10
6.65
90.8
94.8
98.1
Stochastic DDIM
10
6.13
90.4
94.6
97.5
Deterministic DDIM
10
10.9
90.5
95.2
97.7
Unconditional color image generation. On CelebA, we train models for s- (Tab. 4). The two DPDMs use the EDM config (Karras et al., 2022) as well as the Churn sampler; see Tab. 14. For æ=10, DPDM again outperform existing methods by a significant margin. DPDM's synthesized images (see Fig. 16) appear much more diverse and vivid than the baselines' samples.
Table
Noise multiplicity ablation on MNIST for 6=1. See Tab. 11 for extended results.
K
FID CNN-Ace (%)
1
76.9
91.7
2
60.1
93.1
4
57.1
92.8
44.8
94.1
5.2 Ablation Studies
16
36.9
94.2
32
34.8
94.4
Noise multiplicity. Tab. shows results for DPDMs trained with differ- ent noise multiplicity K (using the v-prediction DM config) (Salimans & Ho, 2022). As expected, increasing K leads general trend of improving performance; however, the metrics start to plateau at around K=32.
Diffusion model config. We train DPDMs with different DM configs (sce App. C.1). VP- and VE-based models (Song et al., 2021c) perform poorly for all settings, while for s=0.2 v-prediction significantly outperforms the EDM config on MNIST (Tab. 3). On Fashion-MNIST, the advantage is less significant (extended Tab. 12). For c=(1, 10), the EDM config performs better than v-prediction. Note that the denoiser parameterization for these configs is almost identical and their main difference is the noise distribution p(a) (Fig. 4). As discussed in Sec. 3.2, oversampling large noise levels expected to be especially important. for the large privacy setting (small E), which is validated by our ablation.
Sampling. Tab. 6 shows results for different samplers: deterministic and stochastic DDIM (Song et al., 2021a) as well as the Churn sampler (tuned for high FID scores and downstream accuracy); see App. C.3 for details on the samplers. Stochastic sampling crucial to obtain good perceptual quality, as measured by FID (see poor performance of doterministic DDIM), while it is less important for downstream accuracy. We hypothesize that FID better captures image details that require sufficiently accurate synthesis process. As discussed in Secs. 2.1 and 3.1, stochastic sampling SGD-trained DMs. We also observe that the advantage of the Churn sampler compared to stochastic DDIM becomes less significant as E increases. Moreover, in particular for ==0.2 the FID-adjusted Churn sampler performs poorly on downstream accuracy. This is arguably because its settings sacrifice sample diversity, which downstream accuracy usually benefits from, in favor of synthesis quality (also see samples in App. F.5).
help with that and therefore is particularly important in DP-
12

---

Published in Transactions on Machine Learning Research (08/2023)
6 Conclusions
We propose Differentially Pricate Diffusioo Models (DPDMs), which use DP-SGD to enforce DP guarantees. DMs are strong candidates for DP generative learning due to their robust training objective and intrinsically less complex denoising neural networks. To reduce the gradient variance during training, we introduce noise multiplicity and find that DPDMs achieve state-of-the-art performance in common DP image generation benchmarks. Furthermore, downstream classifiers trained with DPDM-generated synthetic data perform on-par with task-specific discriminative models trained with DP-SGD directly. Note that despite their state-of-the-art results, DPDMs are based on a straightforward idea, this is, to carefully combine DMs with DP-SGD (leveraging the novel noise multiplicity). This "simplicity" is a crucial advantage, as it makes DPDMs a potentially powerful tool that can be casily adopted by DP practitioners. Based on our promising results, we conclude that DMs are an ideal generalive modeling framework for DP generative learning. Moreover, we believe that advancing DM-based DP generative modeling is a pressing topic, considering the extremely fast progress of DM-based large-scale photo-realistic image generation systems (Rombach et al., 2021; Saharia et al., 2022; Ramesh et al., 2022; Balaji et al., 2022). As future directions envision applying our DPDM approach during training of such large image generation DMs, as well as applying DPDMs to other types of data. Furthermore, it may be interesting to pre-train our DPDMs with public data that is not subject to privacy constraints, similar to Harder et al. (2022), which may boost performance. Also see App. H for further discussion on ethics, reproducibility, limitations and more future work.
References
Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Tya Mironov, Kunal Talwar, and Li Zhang. Deep Learning with Differential Privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communicaatinss Securily, pp. 308 318, 2016.
Gergely Acs, Luca Melis, Claude Castelluccia, and Emiliano De Cristofaro. Differentially Private Mixture of Generative Neural Networks. IEEE Transactions on Knowledge and Data Engineering, 31(6):1100-111) 2018.
Rohan Anil, Badih Ghazi, Vineet Gupta, Ravi Kumar, and Pasin Manurangsi. Large-Scale Differentially Private BERT. arXiv:2108.00 01624, 2021.
Martin Arjovsky and Leon Bottou. Towards Principled Methods for Training Generative Adversarial Networks. In International Conference on Learning Representations, 2017.
J. Bailey. The tools of generative art, from flash to neural networks. Art in America, 2020.
Yogesh Balaji, Scungjun Nah, Xun Huang, Arash Vahdat, Jiaming Song, Karsten Kreis, Miika Aittala, Timo Aila, Samuli Laine, Bryan Catanzaro, Tero Karras, and Ming-Yo Yu Liu. eDiff-I: Text-to- Text-to-Image Diffusion Models with Ensemble of Expert Denoisers. arXiv preprint arXiv:2211.011 1.0394,, 2022.
Trene Balelli, Santiago Silva, and Marco Lorenzi. A Differentially Private Probabilistic Framework for Modeling the Variability Across Federated Datasets of Heterogeneous Multi- View Observations. Journal of Machine Learning for Biomedical Imaging, 2022.
Alex Bie, Gautam Kamath, and Guojun Zhang. Private GANs, Revisited. In NewrIPS 2022 Workshop on Synthetic Data for Empowering ML Research, 2022.
Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your Latents: HlighRResolution Video Synthesis with Latent Diffusion Models. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
Andrew Brock, Jeff Donahue, and Karen Simonyan. Large Scale GAN Training for High Fidelity Natural Image Synthesis. In International Conference on Learning Representations, 2019.
Zhiqi Bu, Jialin Mao, and Shiyun Xu. Scalable and Efficient Training of Large Convolutional Neural Networks with Differential Privacy. Advances in Neural Information Processing Systems, 35:38305-383) 2022.
13

---

Published in Transactions on Machine Learning Research (08/2023)
Tianshi Cao, Alex Bie, Arash Vahdat, Sanja Fidler, and Karsten Kreis. Don't Generate Me: Training Differentially Private Generative Models with Sinkhorn Divergence. Advances in Newral Information Processing Systems, 34:12480 12492, 2021.
Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert- Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al. Extracting Training Data from Large Language Models. In 30th USENIX Security Symposium (USENIX Security 21), pp. 2633-2650, 2021.
Dingfan Chen, Tribhuvanesh Orekondy, and Mario Fritz. GS-WGAN: A Gradient-Sanitized Approach for Learning Differentially Private Generators. Advances in Neural Information Processing Systems, 33: 12673-128 12684, 2020.
Jia-Wei Chen, Chia-Mu Yu, Ching- Chia Kao, Tzai- Wei Pang, and Chun-Shiee Lu. DPGEN: Differentially Private Generative Energy-Guided Network for Natural Image Synthesis. In Proceedings of the IEFE/CVF Conference on Computer Vision and Pottern Recognition (CVPR), pp. 8387 8396, June 2022.
Nanxin Chen, Yu Zhang, Heiga Zen, Ron J Weiss, Mohammad Norouzi, and William Chan. WaveGrad: Estimating Gradients for Waveform Generation. In International Conference on Learning Representations, 2021.
Soham De, Leonard Berrada, Jamie Hayes, Samuel L Smith, and Borja Balle. Unlocking High-Accuracy Differentially Private Image Classification through Scale. 2rXiv:2204. 1044..1663 19650, 2022.
Jia Deng, Wei Dong, Richard Sacher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImtgeNet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248 255. leee, 2009.
Prafulla Dhariwal and Alex Nichol. Diffusion Models Beat GANs on Image Synthesis. In Neural Information Processing Systems, 2021.
Tim Dockhorn, Arash Vahdat, and Karsten Kreis. GENIE: Higher-Order Denoising Diffusion Solvers. In Advances in Newral Information Processing Systems, 2022a.
Tim Dockhorn, Arash Vahdat, and Karsten Kreis. Score-Based Generative Modeling with Critically-aamped Langevin Diffusion. In International Conference on Learning Representations, 2022b.
Friedrich Dörmann, Osvald Frisk, Lars Nørvang Andersen, and Christian Fischer Pedersen. Not All Noise is Accounted Equally: How Differentially Private Learning Benefits from Large Sampling Rates. In 2021 IEEE 31st International Workshop on Machine Learning for Signal Processing (MLSP), pp. 6. IEEE, 2021.
Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating Noise to Sensitivity in Private Data Analysis. In Theory of Cryptography Conference, pp. 265-284 Springer, 2006.
Cynthia Dwork, Aaron Roth, et al. The Algorithmic Foundations of Differential Privacy. Foundations and Trends@ in Theoretical Computer Science, 9(3-4)))))))) (1:244111110111 2014.
Lorenzo Frigerio, Anderson Santana de Oliveira, Laurent Gomez, and Patrick Duverger. Differentially Private Generative Adversarial Networks for Time Series, Continuous, and Discrete Open Data. In IFIP International Conference on ICT Systems Security and Privacy Protection, pp. 151-164. Springer, 2019.
Sivakanth Gopi, Yin Tat Lee, and Lukas Wutschitz. Numerical Composition of Differential Privacy, Advances in Neural Information Processing Systems, 34:1 84:11631-11642 2021.
Frederik Harder, Kamil Adamczewski, and Mijung Park. DP-MERF: Differentially Private Mean Embeddings with RandomFeatures for Practical Privacy- Data Generation. In International Conference Artificial Intelligence and Slatistics, pp. 1819 1827. PMLR, 2021.
Fredrik Harder, Milad Jalali Asadabadi, Danica Sutherland, and Mijung Park. Differentially Private Data Generation Needs Better Features. Xrv: 12900, 2022.
14

---

Published in Transactions on Machine Learning Research (08/2023)
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs Trained by Two Time-Scale Update Rule Converge to Local Nash Equilibrium. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems. Curran Associates, Inc., 2017.
Jonathan Ho and Tim Salimans. Classifier-Free Diffusion Guidance. In NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications, 2021.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising Diffusion Probabilistic Models. In Advances in Neural Information Processing Systems, 2020.
Jonathan Ho, Chitwan Saharia, William Chan, David J Flect, Mohammad Norouzi, and Tim Salimans. Cascaded Diffusion Models for High Fidelity Image Generation. or urXix:2106.15280 2021.
Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P. Kingma, Ben Poole, Mohammad Norouzi, David J. Fleet, and Tim Salimans. Imagen Video: High Definition Video Generation with Diffusion Models. arXiv preprint Xrx::2200. 220.0000,,, 2022a.
Jonathan Ho, Tim Salimans, Alexey Gritsenko, Willian Chan, Mohammad Norouzi, and David J. Fleet. Video Diffusion Models. arXiv:2204.03455 2022b.
Aapo Hyvärinen. Estimation of Non-Normalized Statistical Models by Score Matching. Journal of Machine Learning Research, 6:695 709, 2005. ISSN 1532-4435.
Allan Jabri, David Flect, and Ting Chen. Scalable Adaptive Computation for Iterative Goneration.
arXiw:2212.1197 2023.
Myeonghun Jeong, Hyeongju Kim, Sung Jun Cheon, Byoung Jin Choi, and Nam Soo Kim. Diff-TTS: A Denoising Diffusion Model for Text-to-Speech. arXiv preprint 014.:2204.01400, 2021.
Alexia Satcoeur-Marrinennt Ke Li, Rémi Piché- Taillefer, Tal Kachman, and loannis Mitliagkas. Gotta Go Fast When Generating Data with Scoro-Based Models. arXiv:2105. 14080, 2021.
Heewoo Jun, Rewon Child, Mark Chen, John Schulman, Aditya Ramesh, Alec Radford, and Ily Sutskever. Distribution Augmentation for Generative Modeling. In International Conference on Machine Learning, pp. 5006 5019. PMLR, 2020.
Tero Karras, Miika Aittala, Janne Hellsten, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Training Generative Adversarial Networks with Limited Data. Advances in Newral Information Processing Systems, 33:12104-1211 2020a.
Tero Karras, Samuli Laine, Milka Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and Improving the Image Quality of StyleGAN. In Proceedings of the IEEE/CVF Conference on Computer Bision and Pattern Recognition, pp. 8110-8118 2020b.
Tero Karras, Milka Aittala, Samuli Laine, Erik Härkönen, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Alias-Free Generative Adversarial Networks. Advances in Neurol Information Processing Systems, 34: 652-863, 2021.
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the Design Space of Diffusion-Baase Based Generative Models. ar orXim:2206.0032 2022.
Bahjat Kawar, Michael Flad, Stefano Ermon, and Jiaming Song. Denoising Diffusion Restoration Models. arXiv:2201.11793 2022.
Scung Wook Kim, Bradley Brown, Kangxue Yin, Karsten Kreis, Katja Schwarz, Daiqing Li, Robin Rombach, Antonio Torralba, and Sanja Fidler. NeuraFFieldLLDM: Scene Generation with Hicrarchical Latent Diffusion Models. In IEFE Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
15

---

Published in Transactions on Machine Learning Research (08/2023)
Diederik P Kingma and Jimmy Ba. Adam: A Method for Stochastic Optimization. In International Conference on Learning Representations, 2015.
Diederik P Kingma, Tim Salimans, Ben Poole, and Jonathan Ho. Variational Diffusion Models. In Advances in Neural Information Processing Systems, 2021.
Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro, DiffWave: Versatile Diffusion
Model for Audio Synthesis. In International Conference on Learning Representations, 2021.
Alex Krizhevsky. Learning Multiple Layers of Features from Tiny Images. Technical report, University of Toronto, 2009.
Alexey Kurakin, Steve Chien, Shuang Song, Roxana Geambasu, Andreas Terzis, and Abhradeep Thakurta. Toward Training at ImageNet Scale with Differential Privacy. 1.Xiv:2201.12222 2022.
Yann LeCun, Corinna Cortes, and Chris Burges. MNIST handwritten digit database, 2010.
Haoying Li, Yifan Yang, Meng Chang, Huajun Feng, Zhihai Xu, Qi Li, and Yueting Chen. SRDiff: Single Image Reppe--nooottoon with Diffusion Probabilistic Models. 2tt::1044.1495.. 2021.
Xuechen Li, Daogao Liu, Tatsunori B Hashimoto, Huscyin A Inan, Janardhan Kulkarni, Yin-Tar Lee, and Abhradeep Guha Thakurta. When Does Differentially Private Learning Not Suffer in High Dimensions? Advances in Newral Information Processing Systems, 35:28616 221160,, 2022a.
Xuechen Li, Florian Tramer, Percy Liang, and Tatsunori Hashimoto. Large Language Models Can Be Strong Differentially Private Learners. In International Conference on Learning Representations, 2022b.
Seng Pei Liew, Tsubasa Takahashi, and Michihiko Ueno. PEARL: Data Synthesis via Private Embeddings and Adversarial Reconstruction Learning. In International Conference Learning Representations, 2022.
Luping Liu, Yi Ren, Zhijie Lin, and Zhou Zhao. Pseudo Numerical Methods for Diffusion Models on Manifolds.
In International Conference on Learning Representations, 2022.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep Learning Face Attributes in the Wild. In Proceedings of the IEEE International Conference on Computer Vision, pp. 3730 3738, 2015.
Yunhui Long, Suxin Lin, Zhuolin Yang, Carl A Gunter, Han Liu, and Bo Li. Scalable differentially private data generation via private aggregation of teacher ensembles. 2019.
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps. XrXi::2206. 009227, 2022.
Shitong Luo and Wci Hu. Diffusion Probabilistic Models for 3D Point Cloud Generation. In Proceedings of the IEEE/CVE Conference on Compuler Vision and Pallern Recognition (CVPR), 2021.
H Brendan McMahan, Galen Andrew, Ulfar Erlingsson, Steve Chien, Ilya Mironov, Nicolas Papernot, and Peter Kairouz. A General Approach Adding Differential Privacy to Iterative Training Procedures. arXiu:18122 06210, 2018.
Chenlin Meng, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, and Stefano Ermon. SDEdit: Image Synthesis and Editing with Stochastic Differential Equations. arXi::2108.0100 2021.
Lars Mescheder, Andreas Geiger, and Sebastian Nowozin. Which Training Methods for GANs do actually Converge? In Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, 2018.
Iya Mironov. Rényi differential privacy. In 2017 IEEE 30th Computer Security Foundations Symposium (CSF), pp. 263 275. IEEE, 2017.
Ilya Mironov, Kunal Talwar, and Zhang, Rényi Differential Privacy of the Sampled Gaussian Mechanism. arxiv:190s 10530, 2019.
16

---

Published in Transactions on Machine Learning Research (08/2023)
Yisroel Mirsky and Wenke Lee. The Creation and Detection of Deepfakes: A Survey, ACM Comput. Surv., 54(1), 2021.
Thanh Thi Nguyen, Quoe Viet Hung Nguyen, Cuong M. Nguyen, Dung Nguyen, Duc Thanh Nguyen, and Saeid Nahavandi. Deep Learning for Deepfakes Creation and Detection: A Survey, wcXi::199..1177: 2021.
Alexander Quimn Nichol and Prafulla Dhariwal. Improved Denoising Diffusion Probabilistic Models. In International Conference on Machine Learning, 2021.
Art B. Owen. Monte Carlo theory, methods and ezamples. 2013.
Nicolas Papernot and Thomas Steinke. Hyperparameter Tuning with Renyi Differential Privacy. In Interna- tional Conference on Learning Represenlations, 2022.
Nicolas Papernot, Shuang Song, Nya Mironov, Ananth Raghunathan, Kunal Talwar, and Ülfar Erlingsson. Scalable Private Learning with PATE. In International Conference Learning Representations, 2018.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. PyTorch: An Imperative Style, High-Performance Deep Learning Library. Advances in Neural Information Processing Systems, 32, 2019.
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical Text-Conditional
Image Generation with CLIP Latents. orXim:2204.06190 2022.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-Resolution Image Synthesis with Latent Diffusion Models. 2rXic:2112.10722 2021.
Chitwan Saharia, William Chan, Huiwen Chang, Chris A. Lee, Jonathan Ho, Tim Salimans, David J. Fleet, and Mohammad Norouzi. Palette: Image-to-Imas Diffusion Models. arXiv:2111.05822 2021a.
Chitwan Saharia, Jonathan Ho, William Chan, Tim Salimans, David J Fleet, and Mohammad Norouzi. Image Super-Rosolution via Iterative Refinement. arXiv:2104.00 07636, 2021b.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Bureu Karagol Ayan, Sara Mahdavi, Rapha Gontijo Lopes, el Photorealistic Text-to- Image Diffusion Models with Deep Language Understanding. arXio:2205.11458 2022.
Tim Salimans and Jonathan Ho. Progressive Distillation for Fast Sampling of Diffusion Models. In International Conference on Learning Representations, 2022.
Anand Sarwate. Retraction for Symmetric Matrix Porturbation for Differontially-Private Principal Component Analysis, 2017.
Hiroshi Sasaki, Chris G. Willcocks, and Toby P. Breckon. UNIT-DDPM: UNpaired Image Translation with Denoising Diffusion Probabilistic Models. rxXt:22104.03356 2021.
Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, Devi Parikh, Sonal Gupta, and Yaniv Taigman. Mako-A-Video: Text-to- Video Generation without Tixl-Video Data. In The Eleventh International Conference on Learning Representations, 2023.
Jascha Sohl-Diekstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep Unsupervised Learning using Nonequilibrium Thermodynamics. In International Conference on Machine Learning, 2015.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising Diffusion Implicit Models. In International Conference on Learning Representations, 2021a.
Yang Song and Stefano Ermon. Improved Techniques for Training Score-Based Generative Models. Advances in Neural Information Processing Systems, 14:12438-12444 2020.
17

---

Published in Transactions on Machine Learning Research (08/2023)
Yang Song, Conor Durkan, lain Murray, and Stefano Ermon. Maximum Likelihood Training of Score- Basre- Diffusion Models. In Newral Information. Processing Systems (NewrPPS), 2021b.
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-Basod Generative Modeling through Stochastic Differential Equations. In International Conference on Learning Representations, 2021c.
Shun Takagi, Tsubasa Takahashi, Yang Cao, and Masatoshi Yoshikawa. P3GM: Private High-Dimcnsional Data Release via Privacy Preserving Phased Generative Model. In 2021 IEEE 37th International Conference
on Data Engineering (ICDE), pp. 169 180. IEEE, 2021.
Reihaneh Torkzadehmahani, Peter Kairouz, and Benedict Paten. DP-CGAN: Differentially Private Synthetic Data and Label Generation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition Workshops, pp. 0-0, 2019.
Florian Tramer and Dan Boneh. Differentially Private Learning Needs Better Features (or Much More Data). In International Conference on Learning Representations, 2021.
Cristian Vaccari and Andrew Chadwick. Deeplakes and Disinformation: Exploring the Impact of Synthetic Po- litical Video on Deception, Uncertainty, and Trust in News. Social Media + Society, 6(1);2056301120003000, 2020.
Arash Vahdat, Karsten Kreis, and Jan Kautz. Score-based Generative Modeling in Latent Space. In Neural Information Processing Systems (NeurIPS), 2021.
Margarita Vinaroz, Mohanmad-Amin Charusaie, Frederik Harder, Kamil Adamczewski, and Mi Jung Park. Hermite Polynomial Features for Private Data Generation. In International Conference on Machine Learning, pp. 22300 22324. PMLR, 2022.
Boxin Wang, Fan WU, Yunhui Long, Luka Rimanic, Ce Zhang, and Bo Li. DataLens: Scalable Privacy Preserving Training via Gradient Compression and Aggregation. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, pp. 2146-211110 2168, 2021.
Daniel Watson, William Chan, Jonathan Ho, and Mohammad Norouzi. Learning Fast Samplers for Dif- fusion Models by Differentiating Through Sample Quality, In International Conference on Learning Representations, 2022.
Ryan Webster, Julien Rabin, Loic Simon, and Frederic Jurie. This Person (Probably) Exists. Identity Membership Attacks Against GAN Generated Faces. arXiv:2107.00018, 2021.
Han Xiao, Kashif Rasul, and Roland Vollgraf. MNShion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms. (kke::708.077/ 2017.
Zhisheng Xiao, Karsten Kreis, and Arash Vahdat. Tackling the Generative Learning Trilemma with Demoising
Diffusion GANs. In International Conference on Learning Representations, 2022.
Liyang Xie, Kaixiang Lin, Shu Wang, Fei Wang, and Jiayu Zhou. Differentially Private Generative Adversarial Network. arXic:1802. 06739, 2018.
Yasin Yazıcı, Chuan-Sheng Foo, Stefan Winkler, Kim-Hui Yap, Georgios Piliouras, and Vijay Chandrasckhar. The Unusual Effectiveness of Averaging in GAN Training. In International Conference on Learning Representations, 2019.
Hongxu Yin, Arun Mallya, Arash Vahdat, Jose M Alvarez, Jan Kautz, and Pavlo Molchanov. See Through Gradients: Image Batch Recovery via GradInversion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pallern Recognition, pp. 16337-16346, 2021.
Jinsung Yoon, James Jordon, and Mihaela van der Schaar. PATE-GAN: Generating Synthetic Data with Differential Privacy Guarantees. In International Conference on Learning Representations, 2019.
18

---

Published in Transactions on Machine Learning Research (08/2023)
Ashkan Yousefpour, Igor Shilov, Alexandre Sablayrolles, Davide Testuggine, Karthik Prasad, Mani Malek, John Nguyen, Sayan Ghosh, Akash Bharadwaj, Jessica Zhao, Graham Cormode, and Tlya Mironov. Opacus: User-Friendly Differential Privacy Library in PyTorch. In NeurIPS 2021 Workshop Privecy in Machine Learning, 2021.
Da Yu, Saurabh Naik, Arturs Backurs, Sivakanth Gopi, Huseyin A Inan, Gautam Kamath, Janardhan Kulkarni, Yin Tat Lee, Andre Manoel, Lukas Wutschitz, Sergey Yekhanin, and Huishuai Zhang. Differentially Private Fine-tuning of Language Models. In International Conference on Learning Representations, 2022.
Sepanta Zeighami, Ritesh Ahuja, Gabriel Ghinita, and Cyrus Shahabi. A neural database for differentially private spatial range queries. Proceedings of the VLDB Endoument, 15(5):1066-1075 2022.
Xiaohui Zeng, Arash Vahdat, Francis Williams, Zan Gojeic, Or Litany, Sanja Fidler, and Karsten Kreis. LION: Latent Point Diffusion Models for 3D Shape Generation. In Advances in Neural Information Processing Systems, 2022.
Qinsheng Zhang and Yongxin Chen. Fast Sampling of Diffusion Models with Exponential Integrator. 3rXiv:2204.13900 2022.
Linqi Zhou, Yilun Du, and Jiajun Wu. 3D Shape Generation and Completion through Point-Voxel Diffusion. In Proceedings of the IEEE/CVE International Conference on Computer Vision (ICCV), 2021.
Alexander Ziller, Dmitrii Usynin, Rickmer Braren, Mareus Makowski, Daniel Rueckert, and Georgios Kaissis. Medical imaging deep learning with differential privacy. Scientifie Reports, 11(1).1-8, 2021a.
Alexander Ziller, Dmitrii Usynin, Nicolas Remerscheid, Moritz Knolle, Marcus Makowski, Rickmer Braren, Daniel Rueckert, and Georgios Kaissis. Differentially private federated deep learning for multi-site medical image segmentation. arxix:2107. 02586, 2021b.
19

---

Published in Transactions on Machine Learning Research (08/2023)
Contents
Introduction
1
2
Background
3
2.1 Diffusion Models
3
2.2
Differential Privacy
4
3
Differentially Private Diffusion Models
4
3.1
Motivation
5
3.2 Training Details, Design Choices, Privacy
4
Related Work
8
5
Experiments
9
5.1 Main Results
10
5.2 Ablation Studies
6 Conclusions
13
A Differential Privacy and Proof of Theorem 2
21
B DPGEN Analysis
22
C Model and Implementation Details
25
C.1 Diffusion Model Configs
25
C.1.1 Noise Level Visualization
26
C.2 Model Architecture
26
C.3 Sampling from Diffusion Models
26
C.3.1 Guidance
28
C.4 Hyperparameters of Diflerentially Private Diffusion Models
D Variance Reduction via Noise Multiplicity
29
D.1 Proof of Theorem 1
29
D.2 Variance Reduction Experiment
29
D.3 Computational Cost of Noise Multiplicity
29
D.4 On the Difference between Noise Multiplicity and Augmentation Multiplicity
30
E Toy Experiments
31
E.1 Training Details
32
20

---

Published in Transactions on Machine Learning Research (08/2023)
F Image Experiments
33
F.1 Evaluation Metrics, Baselines, and Datasets
33
F.2
Computational Resources
33
F.3
Training DP-SGD Classifiers
F.4
Extended Quantitative Results
34
F.4.1
Noise Multiplicity
34
F.4.2
Diffusion Model Config
34
F.4.3
Diffusion Sampler Grid Search and Ablation
................................
35
F.4.4
Distribution Matching Analysis
................................
36
F.5 Extended Qualitative Results
36
G Additional Experiments More Challenging Problems
36
G.1 Diverse Datasels
36
G.2 Higher Resolution
42
H Ethics, Reproducibility, Limitations and Future Work
43
A Differential Privacy and Proof of Theorem
In this scction, we provide a short proof that the gradicnts released by the Gaussian mechanism in DPDM are DP. By DP, we are specifically refering to the (e, 8D-DP as defined in Definition 2.1, which approximates (e)-DP. For completeness, we state the definition of Rényi Differential Privacy (RDP) (Mironov, 2017): Definition A.1. (Rényi Differential ial Privacy) randomized mechanism M: D - R with domain D and range R satisfies (a, e)-RDP if for any adjacent d, E D:
Da(M(d) Sc.
(8)
where Da is the Rényi divergence of order
Gaussian mechanism can provide RDP according to the following theorem: Theorem 3. (RDP Gaussian mechanism (Mironov, 2017)) For query function f with Sensitivity S
max4. (d) fd' the mechanism that releases f(d) + (00, abr) satisfies (a, (5.555/(220) (232((22))))R)))
Note that any M that satisfics (a, (DDD) also satisfics (c+ log 1/6
a-1 55)DD.
We slightly deviate from the notation used in the main text to make the dependency of variables input data explicit. Recall from the main text that the per-data point loss is computed as average over K noise samples:
1 K 1
K Σ Do + n,kk osk - x:1i, where (losk n (0.))
(9)
In each iteration of Alg. 1, we are given a (random) set of indices B of expected size B with no repeated indices, from which we construct mini-batch {x.) In our implementation (which based on Yousefpour et al. (2021)) of the Gaussian mechanism for gradient sanitization, clipping with norm C, and then divide the clipped gradients by the expected batch size B to obtain the batched gradient Ghatch:
compute the gradient of 4 and apply
Gbatch ((xilie))
Σ clipe (Vol(x,)).
(10)
B
ieB
21

---

Published in Transactions on Machine Learning Research (08/2023)
Finally, Gaussian noise NNO, hp) is added to Gbatch and released as the response Gratch
C
Gbatch ( ((xi)ieR)= Ghatch ( ((x:)ie)) + B
2 2~0(0,,)p11)
(11)
Now, we can restate Theorem 2 as follows with our modified notation:
Theorem 4. For noise magnitude app, dataset d = (x+) and set of (non-repeating) indices IB, releasing Goatch (xx (EB) satisfies
Proof. Without loss of generality, consider two neighboring datasets d= {x and d' dux,, x' $ and mini-batches fxi and x' U (xi where the counter-factual set/ set/batch has additional entry x'. We can bound the difference of their gradients in Lo-norm
|Giaatch (axiooh(((())))))) ((x)iiE) Gbarch (x' U (xilieB)l2
| clpp (Vel(x')) Σ clipe (Vol(x.))
clipo (Vel(x,))
B clipe (Vol(x'))
C B
= B c (Vol(x))ll2
We thus have sensitivity S(Gbatch) G. Furthermore, since z NO Br), (C/B)z ~ (CO, Following standard arguments, releasing Goatch (ix)ie)) Gbatch ((x)) + (C/B)z satisfies (a, a pp RDP (Mironov, 2017).
In practice, we construct mini-batches by sampling the training dataset. for privacy amplification via Poisson Sampling (Mironov et al., 2019), and compute the overall privacy cost of training DPDM via RDP composition (Mironov, 2017). We use these processes as implemented in Opacus (Yousefpour et al., 2021).
For completeness, we also include the Poisson Sampling algorithm in Alg. 2.
Algorithm Poisson Sampling
Input: Index range N, subsampling rate
Output: Random batch indices (of expected size B)
{c, IN Bernoulll)))
B [jje E (1, 1}
B DPGEN Analysis
In this section, we provide detailed analysis of the privacy guarantees provided in DPGEN (Chen et al., 2022).
As a brief overview, Chen et al. (2022) proposes to learn an energy function (o(x) by optimizing the following objective (Chen al. (2022), Eq. 7):
2
x-x o²
[(0:0 σ) - Epix E (x
Vx (x)
2
practice, the first expectation replaced by averaging over examples in a private training set d = {xy z; E YYi 1,...,m), and f is replacod by di = (ā, xx /// for each [1, m] (not be confused with d which denotes the dataset in the DP context), where of the query response produced by data-dependent randomized response mechanism.
22

---

Published in Transactions on Machine Learning Research (08/2023)
We believe that there are three errors in DPGEN that renders the privacy gurantee in DPGEN false. We formally prove the first error in the following section, and state the other two errors which are factual but not. mathematical. The three errors are:
The randomized response mechanism employed in DPGEN has a output space that is only supported (has non-zcro probability) on combinations of its input private dataset. c-difforential privacy cannot be achieved as outcomes with non-zero probability' can have zero probability when the input dataset changed by one element. Furthermore, adversaries observing the output can immediately deduce elements of the private dataset.
The k-nearest neighbor filtering used by DPGEN to reduce the number candidates for the randomized response mechanism is a function of the private data. The likelihood of the k-selected set varies with the noisy image T (line 20 of algorithm in DPGEN), and is not correctly accounted for in DPGEN.
The objective function used train the denoising network in DPGEN depends on both the ground- truth denoising direction and noisy image provided to the denoising network. The noisy image is dependent on the training data, and hence leaks privacy. The privacy cost incurred by using this noisy image is not accounted for in DPGEN.
To prove the first error, we begin with re-iterating the formal definition of differential privacy (DP): Definition B.1. (e-Differential Privacy) A randomized mechanism M D - I with domain D and image I satisfies (s)-DP if for any two adjacent inputs d, d' E DD differing by at most one entry, and for any subset of outputs S C I it holds that
Pr[M(d) E $ e S).
(12)
The randomized response (RR) mechanism is fundamental privacy mechanism in differential privacy. A key assumption required in the RR mechanism is that the choices of random response not dependent private information, such that when respondent draws their response randomly from the possible choices, no privale information is given. More formally, we give the following definition for randomized response over multiple choices?:
Definition B.2. Given a fixed response set Y of size k. Let {a, : 24 E YYic .....,m] be an input datasct. Define "randomized responsc' mochanism RR as:
RR(d) (B((d)==G(x))))
(13)
where,
xi, with probability -
G(x.)
(14)
x E Y Ø4, with probability 1+
A classical result is that the mechanism RR satisfies eDP (Dwork et al., 2014).
DPGEN considers datasets of the form d {a; 24 E i E I....,m) It claims to guarantee differential privacy by applying a stochastic function each element of the dataset defined as follows (Eq. of Chen al. (2022)):
Pr(H(T.) =w]
W =
O = x E
c'
= {a, max(i; - x,y/o; S B,,,, E d) (max over the dimensions of ā; - xz); = = 2, and 8. = =1 2 o20, We first note that H not only function of 24 but also X 24, since its image
where X=
1 "probability over raudomness in the privacy mochanism 2This mechanism analogous the coin flipping mechanism, where the participant first flip biased coin to determine whether they'll answer truthfully lie with probahility lying -< and if they were lie, they then fair dice to determine the response.
23

---

Published in Transactions on Machine Learning Research (08/2023)
is determined by U z4. That is, changes in X will alter the possible outputs of H, independently from the value of T,. We make this dependency explicit in our formulation here-forth. This distinction is important it determines the set of possible outcomes that we need to consider for in the privacy analysis. The authors also noted that 2, is added for training with the denoising objective, not for privacy, so this added Gaussian noise is not essential to the privacy analysis. Furthermore, since k (or equivalently (9) is a hyperparameter that can be tunod, we consider the simpler case where k = m, i.c. X d, as done in the appendix (Eq. 9) by the authors. Thereby we define the privacy mechanism utilized in DPGEN as follows:
Definition B.3. Let = {a+ 24 E R i E 1,...,m. be an input dataset. Define "data dependent. randomized response' M as:
M(d) (H(r,d)))elI,
(15)
where,
Fyi with probability c-+m-1
HI,,,d) = x E d zy with probability e--m-1
(16)
Since the image of (I;;,)) is d, M(d) is only supported on d==3 In other words, the image of M is data dependent, and any outcome O (which are sets of TR" tensors, of cardinality m) that include elements which are not in d would have probability of zero to be the outcome of M(d), i.e. if there exists O and $ d, then Pr[M(d)= = 0]= = 0.
To construct our counter-example, we start with considering two neighboring datasets: the training data d= {ay : I, E , E I.....,.,, and a counter-factual dataset d'= differing in their first element (I1 # z/).. Importantly, since differential privacy requires that the likelihoods of outputs to be similar for all valid pairs of neighboring datasets, we are free to assume that elements of d are unique, i.c. no two rows of d are identical.
= {x1 x E R",,; : z; E R", e.....
Another requirement of differential privacy is that the likelihood of any subsets of outputs must be similar, hence we are free to choose any valid response for the counter-example Thus, letting O denote the outcome of M(d), we choose O = d = Im,. Clearly, by Definition 0.3, this is a plausible outcome of M(d) as it is in the support d However, is not the support of M(d') since the first element æ is not the image of H'd:''; that is (z, d') x1] = 0 for all d'. Privacy protection is violated since any adversary observing O can immediately deduce the participation of 21 in the data release opposed to any counterfactual data. z1-
More formally, consider response set T = {0} C dd and d is the image of M(d), we have
Pr[M(d) E T = Pr[M(d) = O]
(17)
= =Pr(H(01) = 21
(r;H = 2;] (independent dice rolls)
(18)
1=2
e
m
(r.H(z;)= x.]
(apply Definition B.3)
(19)
e9 + m
0 Pr(H( xzl
(20)
1=2
= Pr(H(())) 21
(z. = 21]
(21)
1=2
Pr(M((d = Pr(M(d' = TT.
(22)
We observe that Pr(M(d') T 0, as shown in line 9. Clearly, this result violates 6-DP for all e, which requires Pr[M(d) E T) e Pr(M(d') E T}.
We datasetteepoonenilttion in the sense of repeated cartesian products between sets, Le.. & d
24

---

Published in Transactions on Machine Learning Research (08/2023)
In essence, by using private data to form the response set, we make the image of the privacy mechanism data-dependent. This in turn leaks privacy, since an adversary can immediately rule-out all counter-factual datasets that do not include every element of the response O, as these counter-factuals now have likelihood O. To fix this privacy leak, one could determine a response set a-priori, and use the RR mechanism in Definition B.2 to privalely release data. This modification may not be feasible in practice, since constructing a response set of finite size (k) suitable for images non-trivial. Hence, we believe that it would require fundamental modifications to DPGEN achieve differential privacy.
Regarding error 2, we point out that in the paragraph following Eq. 8 in DPGEN, X is defined the set of & points in d that are closest to ã. when weighted by øj. This means that the membership of X is dependent on the value of ž; Thus, any counter-factual input zú and K with a different set of k nearest neighbors could have many possible outcomes with likelihood under the true input. In essence, this is a more extreme form of data-dependent randomized response where the response set is dependent on both d and 24.-
Regarding error 3, the loss objective in DPGEN (Eq. 1 ((x = V (x))()) includes the term Vz and i is also function of the private data that is yet to be accounted for at all in the privacy analysis of DPGEN. Hence, one would need to further modify the learning algorithm in DPGEN, such that the inputs to the score model are either processed through an additional privacy mechanism, or sampled randomly without dependence on private data.
7 of DPGEN, 1
Regarding justifying the premise that DPGEN implements the data-dependent randomized response mechanism, have verified that the privacy mechanism implemented in the repository of DPGEN (https://githes www.ccccccccccceccecc
is indeed data-dependent:
In line 30 of losses/dsm.pp py:
sample_ ix random. choices (range (k), weights==wiigtt) [o]
randomly selects an index in the range of [0, k - 1], which is then used in line 46,
sample_bute buff append append (samples [sample_ix]] __ix])
to index the private training data and assigned to the output of
sample_brn buff.
Values of this variable are then accessed on line 85 to calculate the "-r" (as x²) term in the objective function (Chen al. (2022), Eq. 7).
C Model and Implementation Details
C.1 Diffusion Model Configs
As discussed in Sec. previous works proposed various denoiser models De, noise distributions p(o), and weighting functions A(o). We refer to the triplet (Do, p, A) as DM config. In this work, consider four such configs: variance preserving (VP) (Song et al., 2021c), variance exploding (VE) (Song et al., 2021c), v-prediction (Salimans & Ho, 2022), and EDM Karras et al. (2022). The triplet for each of these configs can be found in Tab. 7. Note, that we use the parameterization of the denoiser model Dg from (Karras et al., 2022)
Da(x;o) = (o) + Cont (a) (Cin (a)x; Cnoise (n))s
(23)
where Fe is the raw neural network. To accommodate for our particular sampler setting (we require to learn the denoiser model for a E [0.002, 80]: see App. C.3) we slightly modified the parameters of VE and v-prediction. For VE, we changed amin and Fmax from 0.02 to 0.002 and from 100 to 80, respectively. For
In particular, we refer the code commit: (a5
25

---

Published in Transactions on Machine Learning Research (08/2023)
Table 7: Four popular DM configs from the literature.
(Song al, 2021c)
VE (Song al., 2021e) v-prediction (Sallmans Ho, 2022) (Karras al., 2022)
Network and preconditioning
Skip sealing (aip(()
+1)
Ruaal (o (gmna)
Ousput scaling Four(
a
alvito
a diiiall V t
Input scaling (n(a)
1
1/vo" TP
1/y0" oaaa In(o)
Noise cond. (o)les
(M 1)
In([²)
Training
Noise disteibution
(~ M(a.1)
lu(o) (1(((((mmm (n(
(omin fmax))
In(o) NN Paa) (o (o²+o Ho -asss)
Loes weighting
Mo)
1/0²
1/o²
(1 for ("SNR-11" weighting)
Parameters
Ba Amm
Paii
Viec
Pamat -2,, Pua 1.2 ada vi
1000
80
e(1
(6t/2 -1
v-prediction, we changed 6min and émax from 2 arccos V1+e-20 to 2 arccos V1+e-13 and 2 arccos VI+2 to
2 arccos VT+T" respectively. Furthermore, cannot base our EDM models the true (training) data standard deviation data as releasing this information would result in privacy cost. Instead, we set data the standard deviation of uniform distribution between -1 and 1, assuming prior information the modeled image data.
C.1.1 Noise Level Visualization
In the following, we provide details on how exactly the noise distributions of the four configs are visualized in Fig. 4. The reason want to plot these noise distributions is to understand how the different configs assign weight to different noise levels during training through sampling some a's more and others less. However, to be able to make a meaningful conclusion, we also need to take into account the loss weighting X(o).
Therefore, we consider the elfective "imporlaane--eiigueedd distributions plo) XEDM())) where we use the loss weighting from the EDM config reference weighting.
Maa
The X(r)
AnDM((] weightings for VP, VE, v-prediction, and EDM are then, data /(02 2 + o dasa)) dosta/(o2 data)) data (a² + 1)/(0² + dasta); and 1, respectively. Fig. 4 then visualizes the Simporlance-weighee. distributions in log-o space, following Karras et al. (2022) (that way, the final visualized log-o distribution of EDM remains a normal distribution N(Pruean p2 P2
C.2 Model Architecture
We focus on image synthesis and implement the noural network backbone of DPDMs using the DDPM++ architecture (Song et al., 2021c). For class-conditional generation, we add a learned class-embedding to the r-embedding as common practice (Dhariwal & Nichol, 2021). All model hyperparameters and training details can be found in Tab. 8.
C.3 Sampling from Diffusion Models
Let us recall the differential equations we can use to generate samples from DMs:
ODE: dx - (())((()) dt,
(24)
SDE: dx (t) log (t)) dt ((t))2(t) log dt + 22)(t)a(t) dwr.
(25)
Before choosing numerical sampler, we first need to define sampling schedule. In this work, we follow Karras et al. (2022) and use the schedule.
0, (atlle M
i Noll o )
P E (O...., ........ M -1},
(26)
26

---

Published in Transactions on Machine Learning Research (08/2023)
Table Model hyperparameters and training details.
Hyperparameter
MNIST & Pashion-MNIST
CelebA
Model
Data dimensionality (in pixels)
28
32
Residual blocks per resolution
2
Attention resolution(s)
7
8,16
Base channels
32
32
Channel multipliers
1,2.2
1,2,2
EMA rate
0.999
0.999
# of parameters
1.75M
1.80M
Base architecture
DDPM++ (Song et al., 2021c)
DDPM++ (Song et al., 2021c)
Training
# epochs Optimizer
300
300
Adam (Kingma & Ba, 2015)
Adam (Kingmaa & Ba, 2015) 3 - 10-1
Learning rate
3 10-1
Batch size
4096
2048
Dropout
0
Clipping constant C
1
DP-8
10
10 6
with p=7.0, 80 and amin 0..02. We consider two solvers: the (stochastic (1) 1) deterministc (n 0)) DDIM solver (Song et al., 2021a) as well as the stochastic Churn solver introduced in (Karras et al., 2022), for pseudocode see Alg. 3 and Alg. 4, respectively. Both implementations can readily be combined with classifier-free guidance, which described in App. C.3.1, in which case the denoiser (x(x;o) may be replaced by D [x o. y), where the guidance scale w is a hyperparameter. Note that the Churn sampler has four additional hyperparameters which should be tuned empirically (Karras et al., 2022). If not stated otherwise, set M=1000 for the Churn sampler and the stochastic DDIM sampler, and M-50 for the deterministic DDIM sampler.
Algorithm DDIM sampler (Song et al., 2021a)
Input: Denoiser Da(x; r), Schedule (c:).e(o....MM11)
Output: Sample XM Sample Xg N (0, r for n n to M 2 do Evaluate denoiser d., Dx(x;, if Stochastic DDIM then
Xn+1 X. 10m-1-mm (x.. d..) + 22 ont1)anm, zn ~ N(
og
else if Deterministic DDIM then
Xn+1 Xr + @n-1 1n-1-an (xn d.)
end if end for
Return XM D(xx-1,0M-1)
27

---

Published in Transactions on Machine Learning Research (08/2023)
Algorithm Churn sampler (Karras et al., 2022)
Input: Denoiser Da(x;o), Schedule (o.). M-1). Snoise, Schurns Smins Smax
Output: Sample XM Set оM =0
Sample Xo N' (0, (8.
for a to MM do
if E [Smin Smax] then 7/ min( 8 Schurn V2- 1)
else
7/4 =0 end if
Increase noise level on = (1 ya)on
Sample 2. ~ (0, Shoisc 1) and set Nn N. + oz ont
Evaluate denoiser d,, Do (n(nnn and set fn da
Xn+1 = XM + (an+1 Fn if 0n+1 #0 then
Evaluate denoiser d',
ont1) and set fy Xn+i-d,
dnt1
Apply second order correction: Xu+1 Xn + [ - [fa + +fz) end if
end for Return XM
C.3.1 Guidance
Classifier guidance (Song et al., 2021c; Dhariwal & Nichol, 2021) is a technique to guide the diffusion sampling process towards a particular conditioning signal y using gradients, with respect to x, of pre- noisc-conditional classificr p(y)x, Classificr-free guidance (Ho Salimans, 2021), contrast, avoids training additional classifiers by mixing demoising predictions of an unconditional and conditional model, according guidance scale w, by replacing (x or) in the score parameterization se (Do(x; - -x//o* with
(x;o,, (1- (1-w)Do(x;o)
(Do(x;o,)))
(27)
Do(x; a) and Dx(x; y) can be trained jointly; to train Dx o) the conditioning signal y discarded random and replaced by a null token (Ho & Salimans, 2021). Increased guidance scales tend drive samples deeper into the model's modes defined by y at the cost of sample diversity.
C.4 Hyperparameters of Differentially Private Diffusion Models
Tuning hyperparameters for DP models generally induces a privacy cost which should be accounted for (Paper- not & Steinke, 2022). Similar to existing works (De et al., 2022), we neglect the (small) privacy cost associated with hyperparameter tuning. Nonetheless, in this section we want to point out that our hyperparameters show consistent trends across different settings. As a result, we believe our models need little to no hyperparameter tuning similar settings the ones considered in this work.
Model. We the DDPM+ (Song et al., 2021c) architecture for all models in this work. Across all three datasets (MNIST, Fashion-MNIST, and CelebA) found the EDM (Karras al., 2022) config to perform best for c=(1,10.. On MNIST and Fashion-MNIST, we use the v-prediction config for e = 0.2 (not applicable to CelebA).
DP-SGD training. In all settings, we use 300 epochs and clipping constant C=1. We use batch size B-4096 for MNIST and FaSHI and decrease the batch size of CclebA to B=2048 for the sole purpose of fitting the entire batch into GPU memory. The DP noise dpp values for each setup can be found in Tab. 9
28

---

Published in Transactions on Machine Learning Research (08/2023)
Table 9: DP noise ODP used for all our experiments.
E
MNIST
Fashion-MNIST
CelebA
0.2
82.5
82.5
N/A 8.82812
1
18.28125
18.28125
10
2.48779
2.48779
1.30371
DM Sampling. We experiment with different DM solvers in this work. We found the DDIM sampler (Song et al., 2021a) (in particular the stochastic version), which does not have any hyperparameters (without guidance), perform well across all settings. Using the Churn sampler (Karras al., 2022), we could improve perceptual quality (measured in FID), however, out of the five (four without guidance) hyperparameters, only found two (one without guidance) to improve results significantly. We show results for all samplers in App. F.5.
D Variance Reduction via Noise Multiplicity
As discussed in Sec. 3.2, we introduce noise multiplicity to reduce gradient variance.
D.1 Proof of Theorem 1
Theorem. The variance of the DM objective (Eg. (7)) decreases with increased noise multiplicity K as 1/K.
Proof. The DM objective in Eq. (7) a Monte Carlo estimator of the true intractable L2-loss in Eq. (3), using one data sample x; ~ Pdata and K noise-level-noise tuples { Da) ~ p(o)W (0, Replacing expectations with Monte Carlo estimates function over distribution p(k), we have (r/c//((k) K Σ r(k,), where (k,) p(k) (Monte Carlo estimator for expectation of function with respect to distribution p). The Monte Carlo estimate is a noisy unbiased estimator of the expectation [r()] with variance x Var,lr), where Var,/r] is the variance of r itself. This is a well-known fact; see for example Chapter 2 of the excellent book by Owen (2013). This proves that the variance of the DM objective Eq. (7) decreases with increased noise multiplicity K as 1/K.
common practice to ensure numerical tractability. For generic
D.2 Variance Reduction Experiment
In this section, we empirically show how the reduced variance of the DM objective from noise multiplicity leads to reduced gradient variance during training. In particular, we set x; to randomly sampled MNIST image and set the denoiser Do to our trained model on MNTST. We then compute gradients for different. noise multiplicities K. We resample the noise values 1k times (for each K) to estimate the variance of the gradient for each parameter. In Fig. 6, we show the histogram over gradient variance as well as the average gradient variance (averaged over all parameters in the model). Note that the variance of each gradient. is random variable itself (which is estimated using 1k Monte Carlo samples). We find that an increased K leads to significantly reduced training parameter gradient variance.
D.3 Computational Cost of Noise Multiplicity
In terms of computational cost, noise multiplicity is expensive and likely not useful for non-DP DMs. The computational cost increases linearly with K, as the denoiser needs to run K times. Furthermore, in theory, noise multiplicity increases the memory footprint by at least O(K); however, in common DP frameworks, such as Opacus (Yousefpour et al., 2021), which we use, the peak memory requirement is O(K2) compared to non-private training. Recent methods such as ghost clipping (Bu et al., 2022) require less memory, but are currently not widely implemented.
That being said, in DP generative modeling, and DP machine learning more generally, computational cost is hardly ever the bottleneck; the main bottleneck is the privacy restriction. The privacy restriction implies
29

---

Published in Transactions on Machine Learning Research (08/2023)
10 4
3.0
x 10
3
K
Average gradient variance
2.5
K = 8
K = 32
Count
2
2.0
1.5
1.0
10-11
10
10-8
10-3
10-2
Gradient variance
0.5
Noise multiplicity K
32
Figure 6: Variance reduction via noise multiplicity. Inereasing K in moise malliplicity leads to significant. variance reduction of parameter gradient estimates during training (note logarithmic axis in insct). This is an enlarged version of Fig. 3.
only a finite number of training iterations and we need to use that budget training iterations in the most efficient way (this is, using training gradients that suffer from as little noise as possible). Our numerical experiments clearly show that noise multiplicity is a technique shift the privacy-utility trade-off, effectively getting better utility at the same privacy budget, using additional computational cost.
D.4 On the Difference between Noise Multiplicity and Augmentation Multiplicity
Augmentation multiplicity (De et al., 2022) technique where multiple augmentations per image are used to train classifiers with DP-SGD. Image augmentations have also been shown to be potentially helpful in data-limited (image) generative modeling, for example, for autoregressive models (Jun et al., 2020) and DMs (Karras et al., 2022). In stark contrast to discriminative modeling where the data distribution Pdata can simply be replaced by the augmented data distribution and the neural backbone can be left as is, in generative modeling both the loss function and the neural backbone need to be adapted. For example, for DMs (Karras et al., 2022), the standard DM loss (Eq. (3)) is formally replaced by
Ex (R). Panagtta (x( Do + n, x
(28)
where (c) is the distribution over augmentation choices (for example, cropping at certain coordinates or other image transformations or perturbations), and Paugdata (x x; e) is the distribution over augmented images x given the original dataset images and the augmentation c. Importantly, note that the neural backbone Dg also needs to be conditioned on the augmentation choice as, inference time, we generally only want to generate "clean images" with c(x) (no angmentation).
While noise multiplicity provably reduces the variance of the standard diffusion loss (sce Theorem 1), augmen- lation multiplicity, that is, averaging over multiple augmentations for a given clean image x, only provably reduces the variance the augmented diffusion loss, which by definition more noisy due to the additional expectations. Furthermore,
not obvious how minimizing the augmented diffusion loss relates minimizing the true diffusion loss. In contrast noise multiplicity, augmentation multiplicity does not provably reduce the variance of the original diffusion loss; rather, it is data augmentation technique for enriching training data.
30

---

Published in Transactions on Machine Learning Research (08/2023)
(a) Data Pinta
(b) Samples from DM.
(c) Samples from GAN.
Figure Mixture of Gaussians: data distribution and (1M) samples from a DM as well as a GAN. Our visualization based on the highhisogr,mm which shows single data points as black dots.
Pointing out the orthogonality of the two ideas again, note that noise multiplicity is still applica- ble for the above modified augmented diffusion loss objective. Furthermore, we would like to point out that noise multiplicity is applicable for DPDMs in any domain, beyond images; in contrast, data augmentations need to be handcrafted and may not readily available in all fields.
E Toy Experiments
In this section, we describe the details of the toy experiment from paragraph (ii) Sequential denoising in Sec. 3.1. For this experiment, we consider a two-dimensional simple Gaussian mixture model of the form
Pdaaaa (x) =
pl (x):
(29)
k=1
where (x) = -N(x; px: of) and
a/22
0
μ1
12
a/2
13
0
0/22
a/22
p4
a/2
145
0
146
a/2)
a/2
a
147
148
a/2
14g
0
where 1/25 and a = 1/V2. The data distribution is visualized in Fig. 7a.
Fitting. Initially, we fitted DM as well as a GAN to the mixture Gaussians. The neural networks of the DM and the GAN generator use similar ResNet architectures with 267k and 264k (1.1% smaller) parameters, respectively (see App. E.1 for training details). The fitted distributions are visualized in Fig. 7. In this deterministic DDIM (Alg. 3) (Song et al., 2021a), numerical solver for the Probability Flow ODE (Fq. (1)) (Song et al., 2021c), with 100 neural function evaluations (DDIM-100) as the end-to-end multi-step synthesis process for the DM. Even though our visualization shows that the DM clearly fits the distribution better (Fig. 7), the GAN does not do bad either. Note that our visualization ion is based on the historaaa of the sampling distributions, and therefore puts significant emphasis on single data point outliers.
experiment,
We provide second method assess the fitting: In particular, we measure the percentage of points (out of 1M samples) that are within h-standard deviation vicinity of any of the nine modes. A point is said to be within h-standard deviation vicinity of the mode μ4 if |xx pall < hog- We present results for this metric in Tab. 10 for 1-11, 3, 4, 4,J, Note that any mode is at least 12.5 standard deviations separated to the next mode, and therefore no point can be in the h-standard deviation vicinity of more than two modes for < 6.
The results in Tab. 10 indicate that the GAN slightly too sharp, that is, it puts too many points within the 1- and 2-standard deviation vicinity of modes. Moreover, for larger h, the result in Tab. 10 suggests that the
31

---

Published in Transactions on Machine Learning Research (08/2023)
Table 10: h-standard deviation vicinity metric as defined the paragraph Fitting of App. E.
h
Data
DM
GAN
39.4
37.2
56.8
2
86.5
83.3
95.3
3
98.9
97.7
98.9
4
100
99.8
99.3
100
100
99.6
100
100
99.9
samples in Fig. 7c that appear to "connect" the GAN's modes are heavily overemphasized- samples actually represent less than 1% of the total samples; 99.3% of samples are within 4-standard deviation vicinity of a mode while modes are at least 12.5 standard deviations separated.
Complexity. Now that we have ensured that both the GAN as well as the DM the target distribution reasonably well, we can measure the complexity of the DM denoiser D, the generator defined by the GAN, as well as the end-to-end multi-step synthesis process (DDIM-100) of the DM. In particular, we measure the complexity of these functions using the Frobenius norm of the Jacobian (Dockhorn et al., 2022b). In particular, define
JF(d) Exxp(xxor) V Vx o)
(30)
Note that the convolution of mixture of Gaussian with ii.d. Gaussian noise is simply the sum of the convolution of the mixture components, i.e.,
ox (Pdata *N (0, o0, (x)
(31)
N(x: uui of +²²).
(32)
We then compare Tr(a) with the complexity of the GAN generator (S1) and the end-to-end synthesis process of the DM (82). In particular, define
JF -
ie [1,2).
(33)
We want to clarify that for S2 we do not have to backpropagate through an ODE but rather through its discretization, i.c., deterministic DDIM with 100 function evaluations (Alg. 3), since that is how define the end-to-end multi-step synthesis process of the DM in this experiment. Furthermore, we chose the latent space of the GAN to be two-dimensional such that (x) 2R2x2 for both the GAN and the DM; this ensures a fair comparison. The final complexities are visualized in Fig. 2.
E.1 Training Details
DM training. Training the diffusion model is very simple. We use the EDM config and train for 50k iterations (with batch size B-256) using Adam with learning rate 3 10-4, We use an EMA rate of 0.999.
GAN training. Training GANs two-dimensional mixture of Gaussians is notoriously difficult (see, for example, Sec. 5.1 in (Yazıcı et al., 2019)). We experimented with several setups and found the following to perform well: We train for 50k iterations (with batch size B-256) using Adam with learning rale 3 10-4 and (3,=0.0, 82- = 0.9) for both the generator and the discriminator. Following YaNııı et al. (2019), we use EMA (rate of 0.999 in particular, we use twice as many hidden layers in the discriminator's ResNet. Furthermore, we use ReLU and LeakyReLU for the generator and the discriminator, respectively.
the DM). We found crucial to make the discriminator bigger than the generator;
32

---

Published in Transactions on Machine Learning Research (08/2023)
F Image Experiments
F.1 Evaluation Metrics, Baselines, and Datasets
Metrics. We measure sample quality via Fréchet Inception Distance (FID) (Heusel et al., 2017). We follow the DP generation literature and use 60k generated samples. The particular Inception-v3 model used for FID computation is taken from Karras et al. (2021) On MNIST and Fashion-MNIST, we follow the standard procedure of repeating the channel dimension three times before feeding images into the Inception-v3 model.
On MNIST and Fashion-MNISST, we additionally assess the utility of generated data by training classifiers synthesized samples and compute class prediction accuracy real data. Similar to previous works, we consider three classifiers: logistic regression (Log Reg), MLP,, and CNN classifiers. The model architectures are taken from the DP-Sinkhorn repository (Cao ct al., 2021).
For downstream classifier training, we follow the DP generation literature and use 60k synthesized samples. We follow Cao et al. (2021) and split the 60k samples into a training set (90%) and a validation set (remaining 10%). We train all models for 50 epochs, using Adam with learning rate 3 10-4. We regularly save checkpoints during training and use the checkpoint that achieves the best accuracy on the validation split for final evaluation. Final evaluation is performed real, non-synthetic data. We train all models for 50 epochs, using Adam with learning rate 3 10-4.
Note that we chose to only use 60k synthesized samples to follow prior work and therefore be able to compare to baselines in a fair manner. That being said, during this project we did explore training classifiers with more samples but did not find any significant improvements in downstream accuracy. We hypothesize that 60k samples are enough to accurately represent the underlying learned distribution by the DPDM and to train good classifiers MNIST /FaSSioMMoSSTT... We believe that more detailed study on how many samples are needed to get a certain accuracy is an interesting avenue for future work.
Baselines. We run baseline experiments for PEARL (Liew et al., 2022). In particular, we train models for c= 1, 10} on MNIST and Fashion-MNIST. We confirmed that our models match the performance reported in their paper. In fact, our models perform slightly better (in terms of the LeNet-FID metric Liow et al. (2022) uses). We then follow the same evaluation setup (see Metrics above) as for our DPDMs. Most, importantly, we use the standard Inception network-based FID calculation, similarly as most works in the (DP) image generative modcling literature.
Datasets. We use three datasets in our main experiments: MNIST (LeCun et al., 2010), MaShion-MNIST (Xiao et al., 2017) and CelebA (Liu et al., 2015). Furthermore, we provide initial results on CIFAR-10 (Krizhevsky, 2009) and ImageNet (Deng et al., 2009) (as well as ColebA on a higher resolution); see App. G.
We would like to point out that these dataset may contain multiple images per identity (e.g. person, animal, etc.), whereas our method, as well as all other baselines in this work, considers the per-image privacy guarantee. For an identity with & images in the dataset, a model with (e,d) per-image DP affords (ke, (e(t=1)e to the individual according to the Group Privacy theorem (Dwork et al,, 2014). We leave more rigorous study of DPDMs with Group Privacy to future rescarch and note that these datascts currently simply serve as benchmarks in the community. Nenetheless, we believe that it is important to point out that these datasets do not necessarily serve as a realistic test bed for vergi DP generative models in privacy critical applications.
F.2 Computational Resources
For all experiments, we use an in-house GPU cluster of V100 NVIDIA GPUs, On eight GPUs, models on MNIST and Fashion-MNIST trained for roughly one day and models on CelebA for roughly four days. We tried to maximize performance by using a large number of epochs, which results in a good privacy-utility trade-off, as well as high noise multiplicity; this results in relatively high training time (when compared to existing DP generative models). Using a smaller noise multiplicity K decreases computation, although generally at the cost of model performance; see also App. D.3.
5 "netpee// http:///tpp....//dd//////.o comm http://///////////////// hww.rrcch/w//hhhhhh/ccoc /venskons/1 /1 /oar////ggn///////sssss
33

---

Published in Transactions on Machine Learning Research (08/2023)
Table 11: Noise multiplicity ablation on MNIST and Pashion-MNIST.
MNIST
Cashion-MNIST
K
Acc
(%)
Ace (%)
FID
FID
Log Reg
MLP
CNN
Log Reg
MLP
CNN
76.9
84.2
87.5
91.7
72.5
76.0
76.3
75.9
2
60.1
84.8
88.3
93.1
61.4
76.7
77.0
77.4
4
57.1
85.2
88.0
92.8
61.1
76.7
77.2
77.0
8
44.8
86.2
89.2
94.1
58.2
75.2
76.3
77.4
16
36.9
86.0
89.8
94.2
58.5
77.0
77.4
78.8
32
34.8
86.8
90.1
94.4
57.7
76.4
77.0
77.1
Table 12: DM config ablation.
MNIST
Mishion-MNISS
Method
DP-8
Ace (%)
Ace (%)
FID
FID
Log Reg
MLP
CNN
Log Reg
MLP
CNN
VP (Song al., 2021c)
0.2
197
23.1
25.5
24.2
146
49.7
51.6
51.7
VE (Song et al., 2021c)
0.2
171
17.9
15.4
13.9
178
22.2
27.9
49.4
V V-prediction (Salimans & Ho, 2022)
0.2
97.8
80.2
81.3
84.4
115
71.3
70.9
71.8
EDM (Karras et al., 2022)
0.2
119
62.4
67.3
49.2
93.5
64.7
65.9
66.6
VP (Song al., 2021c)
82.2
59.4
69.3
72.6
73.4
70.4
72.7
VE (Song et al., 2021c)
165
17.9
20.5
26.0
156
30.7
36.0
49.8
V V-prediction (Salimans & Ho, 2022)
34.8
86.8
90.1
94.4
57.7
76.4
77.0
77.1
EDM (Karras et al., 2022)
34.2
86.2
90.1
94.9
47.1
77.4
78.0
79.4
(Song al., 2021c)
10
12.3
88.8
94.1
97.0
22.3
VE (Song al., 2021c)
10
88.6
48.0
56.9
63.8
83.2
69.0
70.4
75.4
V V-prediction (Salinans & Ho, 2022)
10
7.65
90.4
94.4
97.7
23.1
82.0
83.7
85.5
EDM (Karras et al., 2022)
10
6.13
90.4
94.6
97.5
17.4
82.6
84.1
86.2
F.3 Training DP-SGD Classifiers
We train classifiers MNIST and Fashion-MNIST using DP-SGD directly. We follow the setup used for training DPDMs, in particular, batchsize B 4096, 300 epochs and clipping constant C' = 1. Recently, De et al. (2022) found EMA to be helpful in training image classificrs: we follow this suggestion and use an EMA rate of 0.999 (same rate as used for training DPDMs)
F.4 Extended Quantitative Results
In this section, we show additional quantitative results not presented in the main paper. In particular, we present extended results for all ablation experiments.
F.4.1 Noise Multiplicity
In the main paper, we present noise multiplicity ablation results on MNIST with s=1 (Tab. 5). All results for MNIST and Fashion-MNIST on all three privacy settings (==(0.2,1,100) can be found in Tab. 11.
F.4.2 Diffusion Model Config
In the main paper, we present DM config ablation results on MNIST with ==0.2 (Tab. 5). All results for MNIST and Fashion-MNIST on all three privacy settings (e= 10)) be found in Tab. 12.
34

---

Published in Transactions on Machine Learning Research (08/2023)
Table 13: Diffusion sampler comparison. We compare the Churn sampler (Karras et al., 2022) to stochastic and determistic DDIM (Song al., 2021a).
MNIST
Beshion-MMNSS
Sampler
DP-s
Ace (%)
Ace (%)
FID
FID
Log Reg
MLP
CNN
Log Reg
MLP
CNN
Churn (FID)
0.2
61.9
65.3
65.8
71.9
78.4
53.6
55.3
57.0
Churn (Ace)
0.2
104
81.0
81.7
86.3
128
70.4
71.3
72.3
Stochastic DDIM
0.2
97.8
80.2
81.3
84.4
115
71.3
70.9
71.8
Deterministic DDIM
0.2
120
81.3
82.1
84.8
132
71.5
71.6
71.8
Churn (FID)
23.4
83.8
87.0
93.4
37.8
71.7
73.6
Churn (Acc)
1
35.5
86.7
91.6
95.3
51.4
76.3
76.9
79.4
Stochastic DDIM
34.2
86.2
90.1
94.9
47.1
77.4
78.0
79.4
Deterministic DDIM
1
50.4
85.7
91.8
94.9
60.6
77.5
78.2
78.9
Churn (FID)
10
5.01
90.5
94.6
97.3
18.6
80.4
81.1
84.9
Churn (Ace)
10
6.65
90.8
94.8
98.1
19.1
81.1
83.0
86.2
Stochastic DDIM
10
6.13
90.4
94.6
97.5
17.4
82.6
84.1
86.2
Deterministic DDIM
10
10.9
90.5
95.2
97.7
19.7
81.9
83.9
86.2
Table 14: Best Churn sampler settings for FID metric.
MNIST
Fashion-MNIST
CelebA
Parameter
2 002
8=1
E=
5=0.2
E=10
s=1
s=10
0
0.25
2
1
0.25
N/A
N/A
Sehurn
200
100
50
150
50
25
200
50
Smin
0.01
0.05
0.05
0.02
0.025
0.2
0.005
0.005
Smax
50
50
50
10
50
50
50
Snoise
1
1
F.4.3 Diffusion Sampler Grid Search and Ablation
Churn sampler grid search. We run a small grid search for the hyperparameters of the Churn sampler (together with the guidance weight for classifter-free guidance). For MNIST and Fashion- MNIST on ==0.2 we run two-stage grid search. Using Smmm=0.05, =00 and Suoise=1, =1, which we found to be sensible starting values, we ran an initial grid search over (0, 0. 125, 0.25, 0.5, 1.0, 2.0} and Sehurn (0, 5, 10, 25, 50, 100, 150, 200}, which we found be the two most critical heperparameters of the Churn sampler. Afterwards, we ran a second grid search over -no 1.005), 5min= 0.02, 0.05, 0.1, 0.2), and mmax= 50, 80} using the best (w, Schura) setting for each of the two models. For MNIST and Fashion- single full grid search over w= 0.25, 0.5, 1.0, Sehum 25, 50, 100), and (0.05 0.05, 0.1, 0.2) while setting =1. For CelebA, on both e-1 and ==10, we also ran single full grid search over Schum=(50, 100, 150, 200), and Smi = 0.05} while setting Smmis. The best settings for FID metric and downstream CNN accuracy can be found in Tab. 14 and Tab. 15, respectively.
MNIST on == (1,10) 10),
Throughout all experiments we found two consistent trends that are listed in the following:
optimizing for FID, set Schura relatively high and Smin relatively small. Increase Schurn and decrease Smin as decreased.
optimizing for downstream accuracy, set Schurn relatively small and Smin relatively high.
Sampling ablation. In the main paper, we present sampler ablation for MNIST (Tab. 6). Results for Fashion-MNIST (as well as) MNIST can be found in Tab. 13.
35

---

Published in Transactions on Machine Learning Research (08/2023)
Table 15: Best Churn sampler settings for downstream CNN accuracy.
MNIST
Pashion-MNIST
Parameter
0.2
s=1
e-10
6-0.2
e=1
e- 10
W
0.125
0
0
0.125
Schurn
10
10
5
10
Smin
0.2
0.1
0.025
0.02
0.025
0.1
Smax
10
50
50
80
50
50
Snoise
1.005
1
1
1.005
Table 16: Distribution matching analysis for MNIST using downstream CNN accuracy.
Class
2
4
6
7
9
e=10 10
98.5
98.7
98.7
98.4
99.1
99.0
98.2
97.7
98.1
96.1
e=1
95.4
98.9
96.7
96.1
96.1
97.3
96.6
91.3
92.3
93.2
e 0.2
81.9
96.0
80.2
82.3
87.3
85.2
90.7
86.9
83.2
83.5
F.4.4 Distribution Matching Analysis
We perform distribution matching analysis on MNIST using the CNN classifier, that is, computing per-class downstream accuracies at different privacy levels. In Tab. 16, we can see that with increased privacy (lower e) classification performance degrades roughly similar for most digits, implying that our DPDMs learn a well-balanced distribution and cover all modes of the data distribution faithfully even under strong privacy. The only result that stands out to us is that class 1 appears significantly casier than all other classes for e=0.2. However, that may polentially be due to the class in MNIST being "line" which may be easy to classify by CNN even if training data is noisy.
F.5 Extended Qualitative Results
In this section, we show additional generated samples by our DPDMs. On MNIST, see Fig. 8, Fig. 9, and Fig. 10 for ==10, æ=1, and ==0.2, respectively. On Fashion-MNIST, see Fig. 11, Fig. 12, and Fig. 13 for e=10, e=1, and e=0.2, respectively. On CelebA, see Fig. 14 and Fig. 15 for e=10 and 6=1, respectively. For visual comparison of our CelebA samples to other works in the literature, Fig. 16.
G Additional Experiments on More Challenging Problems
G.1 Diverse Datasets
We provide results for additional experiments on challenging diverse datasets, namely, CIFAR-10 (Krizhevsky, 2009) and ImageNet (Deng et al., 2009) (resolution 32x32), both in the class-conditional setting similar to our other experiments on MNIST and Fashion-MNIST.. To the best of our knowledge, we are the first to attempt. pure DP image generation on ImagenNet.
For both experiments, we the same neural network architecture for CelebA (32x32) in the main paper; model hyperparameters in Tab. 8. On CIFAR-10,
train for 500 epochs using noise multiplicity K 32 under the privacy setting (s 10, 8 10 10-5). In ImageNet, we train for 100 epochs using noise multiplicity = 8 under the privacy setting (a = 10, 5 = 7 170-)) training for longer (or using larger K) was not possible on ImageNet due to its sheer size. We achieve FIDs of 97.7 and 61.3 for CIFAR-10 and ImageNet, respectively. No previous works reported FID scores on these datasets and for these privacy settings, but we hope that our scores can serve as reference points for future work. In Fig. 17, we show samples for both datascts from our DPDMs and visually compare an existing DP generative modeling work on CIFAR-10, DP-MERF (Harder et al., 2021). Our DPDMs cannot learn clear objects; however, overall image/pixel statistics seem to be
36

---

Published in Transactions on Machine Learning Research (08/202))
0
2
3
4
5
6
7
9
Q
3
7
9
0
2
3
4
5
6
7
8
9
0
2
3
4
8
6
7
9
0
2
5
6
7
0
2
3
4
5
6
7
0
3
4
5
6
7
8
Q
1
4
5
6
7
8
9 9
1
4
5
8
9
/
3
q 3
8
O
1
4
8 9 9 9
O
4
13
4
7
8
9
0
1
2
4
5
6
0
2
1
6
7
0
1
2
3
5
7 6
8 8 8
0
2
3
4
5
9
0
1
2
3
4
5
7
9 9
0
1
2
3
4
5
6
7 7
3
9
0
2
3
4
6
6
7
/
2
3
5
6
?
Q
3
4
5
6
7 7
8 9 9
O
2
3
5
K
7
9 9
0
3
3
4
8
6
0
3
3
4
5
6
7
8
0
2
3
4
5
6
7
8 8
0
2
4
5
6
0
2
3
4
6
7
7 9 9
0
1
2
3
5
6
7
8
9 9
/
2
3
4
6
7
8
0
1
4
3
4
5
6
7
8
0
4
4
5
6
7
8
C
1
2 2
3
4
5
6
8
9
0
1
2
4
S
6
7
0 1
3
4
5
6
7
8
0
2
3
4
5
6
7
9 9
2
3
5
6
7
9 9
2
3
4
5
6
7
5
17
2
3
4
5
6
7
8
0
/
2
3
4
5
6
8
0
2
3
4
5
6
7
8
9
Figure 8: Additional images generated by DPDM on MNIST for 8=10 using Churn (FID) (top left), Churn
(Acc) (top right), stochastic DDIM (bottom left), and deterministic DDIM (bottom right).
0
A
3
4
5
6
7
9
5
48
7
9
0
2
3
9
6
6
7
8
9
0
J
3
4
5
6
7
8
7
0
3
4
5
6
7
4
Q
[
2
3
4
5
4
7
0
2
3
4
6
6
7
8
7
O
2
3
5
6
7
2
1
2
3
4
5
6
7
8
9 9 9 9
/
3
5
6
7
7
9
O
/
2
4
5
6
7
8
O 1
83
4
5
6
7
7
9
0
2
3
5
6
7
0 2
5
à
7
9
9
1
2
3
4
5
6
7
8
0
1
2
3
4
6
4
7
9
0
1
1
4
5
6
9
0
(
2
3
4
5
6
7
3
9
0
~
3
4
5
6
7
8
O
4
7
2
3
5
&
7
8
+
0
Q
5
4
5.
l
7 7
9 3
O
2
3
5 $
&
7
9 9
0
3
3
4
5
6
? g
O
2
3
f 6 4
7 m
8
0
0(23
4
5
4
7
Q
2
7
4
5
7
0
2
5
6
7
8
7 9
3
1 1
2
15
7
9
/ 3
5
6
7
7 7
O
1
n
8
7
0 11436
4
5
4
9
G 11333444
4
4
0 1 2
5
5
7
7
9
0 1
2
8
4
5
9
/ 2
3
5
7
9
4
7
3
5
6
9
1
2 3
4
5
9
2
3
5
6
7
8
9
0
/ 3
8
4
9
2
3
4
5 &
9
Figure 9: Additional images generated by DPDM on MNIST for ==1 using Churn (FID) (top left), Churn
(Acc) (lop right), stochastic DDIM (bollom left), and deterministic DDIM (bollom right).
37

---

Published in Transactions on Machine Learning Research (08/2023)
0 2 4 /. 6
E
D
2
0
4
2
2
9
0 1 1 7 9
T
{
L P 2
3
+
4
07:34
1773426755
O
3 ] 3
1/
3
6
P
7
9
0173y6670
0
<
&
7
O 01220667F1
1 7
4
4
C 012346772
D
3
4
5
&
F
0 [ /
Q
2
4
2
7 7 7
Y
0 2 1733467711
0
1
7
3
4
9
3
1
0 1 0
0
1
0
2
1
8
9
0
C
/
R
3
4
5
4
7
B
0
Q
3
/.
4
7 7 7
7 7 マ
7
D
2
3 3 7
4
1 2
%
0
{ J
3
4
4
J.
7 5'
9
C [
3
4
7
0
2
5
4
r
?
0 1 7 3
<
f.
7
8
7
7
4
7
7
/ 3
]
1
9
6
7
O
7
3
K
4
.7
4
7
D 1
A
3
4
H
i
7
9
G
5
4
7
4
7
4
5
@
1
2
3
1
8
Q
{
3
4
5
8
?
7
8
/
3
3 2
4
G
3 7
4
7
3
7
5
7
0
7
m
6
9
4
7.
7
J
#
7 4
2
5
3
1
8
7
8
2
5 5
5
3
8
Figure 10: Additional images generated by DPDM on MNIST for c=0.2 using Churn (FID) (top left), Churn
(Acc) (top right), stochastic DDIM (bottom left), and deterministic DDIM (bottom right).
Figure 11: Additional images generated by DPDM on Fashion-MNIST for ==10 using Churn (FID) (top left),
Churn (Ace) (lop right), stochastic DDTM (bollom left), and deterministic DDIM (boltom right).
38

---

Published in Transactions on Machine Learning Research (08/2023)
Figure 12: Additional images generated by DPDM on Fashion-MNIST for 6=1 using Churn (FID) (top left),
Churn (Ace) (top right), stochastic DDIM (bottom left), and deterministic DDIM (bottom right).
Figure 13: Additional images generated by DPDM on Fashion-MNIST for c=0.2 using Churn (FID) (top
left), Churn (Acc) (lop right), stochastic DDIM (boltom left), and deterministic DDIM (bollom right).
39

---

Published in Transactions on Machine Learning Research (08/2023)
Figure 14: Additional images generated by DPDM on CelebA for ==10 using Churn (top), stochastic DDIM (middle), and deterministic DDIM (bottom).
40

---

Published in Transactions on Machine Learning Research (08/2023)
Figure 15: Additional images generated by DPDM on CelebA for s=1 using Churn (top), stochastic DDIM (middle), and deterministic DDIM (bottom).
41

---

Published in Transactions on Machine Learning Research (08/2023)
Figure 16: CelebA images generated by DataLens (1st row), DP-MEPF (2nd row), DP-Sinkhorn (3rd row), and our DPDM (4th row) for DP-g=10.
(a) DPDM (ours) (ImageNet)
(b) DPDM (ours) (CIFAR-10)
Figure 17: Additional experiments on challenging diverse datasets. Samples from our DPDM on ImageNet and CIFAR-10, as well as CIFAR-10 samples from DP-MERF (Harder al., 2021) (c).
captured correctly. In contrast, the DP-MERF baseline collapses entirely. We are not aware of any other works tackling these tasks. Hence, we believe that DPDMs represent major step forward.
G.2 Higher Resolution
We provide results for additional experiments CelebA at higher resolution (64x64). To accommodate the higher resolution, we added an additional in roughly 11% increase in the number of paramcters, from 1.80M to 2.00M parameters. The only row that changed in the CelebA model hyperparameter table (Tab. 8) is the one about the channel multipliers. It is adapted from (1,2,2) to (1,2,2,2). We train for 300 epochs using = 8 under the privacy setting ===10, 10-9). We achieve an FID of 78.3 (again, for reference; no previous works reported quantitative results on this task). Fig. 18, we show samples and visually compare to existing DP generative modeling work CelebA at 64x64 resolution. Although the faces generated by DPDM somewhat. distorted,
layer to the U-Net, which results
42

---

Published in Transactions on Machine Learning Research (08/2023)
(a) DPDM (ours)
(b) DataLens (Wang et al., 2021)
Figure 18: Additional experiments on CelebA at higher resolution (64x64). Samples from our method and DataLens (Wang et al., 2021).
the model overall is able to clearly generate face-like structures. In contrast, DataLens generates incoherent very low quality outputs. No other cxisting works tried generating 64x64 CelobA images with rigorous DP guarantees, to the best of our knowledge. Also this experiment implies that DPDMs can be considered a major step forward for DP generative modeling.
H Ethics, Reproducibility, Limitations and Future Work
Our work improves the state-of-the-art in differentially private generative modeling and we validate our proposed DPDMs on image synthesis benchmarks. Generative modeling images has promising applications, for example for digital content creation and artistic expression (Bailey, 2020), but can in principle also be used for malicious purposes (Vaccari & Chadwick, 2020; Mirsky & Lee, 2021; Nguyen et al., 2021). However, differentially private image generation methods, including our DPDM, are currently not able to produce photo-realistic content, which makes such abuse unlikely.
As discussed in Sec. 1, a severe issue in modern generative models is that they can easily overfit to the data distribution, thereby closcly reproducing training samples and leaking privacy of the training data. Our DPDMs aim lo rigorously address such problems via the well-established DP framework and fundamentally protect the privacy of the training data and prevent overfitting to individual data samples. This is especially important when training generative models diverse and privacy-sensitii data. Therefore, DPDMs potentially act we hope will benefit the broader machine learning community. Note, however, that although DPDM provides privacy protection in generative learning, information about individuals cannot be eliminated entirely, as no useful model can be learned under (P=(6=0,6-))) This should be commmicated clearly to dataset participants.
effective medium for data sharing without needing to worry about data privacy, which
An important question for future work is scaling DPDMs to (a) larger datasets and (b) more complicated, potentially higheroresolution image datasets. Regarding (a), recall that an increased number of data points leads to less noise injection during DP-SGD training (while keeping all other parameters fixed). Therefore, we believe that DPDMs should scale well with respect to larger datasets; see, for example, our results on ImageNet, a large datasel, which are to some degree better than the results on CIFAR-10, a relatively small dataset (App. G). Regarding (b), however, scaling to more complicated, potentially nigherr-esolution datascts is challenging the number of data points is kept fixed. Higher-resolution data requires larger neural networks, but this comes with more parameters, which can be problematic during DP training (see Sec. 3.2). Using parfienter-effccent architectures for diffusion models may be promising; for instance, see concurrent.
43

---

Published in Transactions on Machine Learning Research (08/2023)
work by Jabri et al. (2023). Generally, we believe that scaling both (a) and (6) are interesting avenues for future work.
To aid reproducibility of the results and methods presented in our paper, we made source code to reproduce all quantitative and qualitative results of the paper publicly available, including detailed instructions. Moreover, all training details and hyperparameters are already described in detail in this appendix, in particular in App. C.
44


Reviews:
[{"review_id": "aJeswcz5xQ", "reviewer": "Reviewer_6kWp", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The paper motivates why using DP-di!usion models would be better than DP-GANs very well\n2. The idea of noise multiplicity is very nice and e!ective\n3. The finding that DP generated data results in a classifier that is even better than the DP-sgd trained classifiers is very promising.\n\n\n\nWeaknesses:\n1. Limitations of the proposed method are not really discussed. I wonder how well the method scales to larger class classification problems, with more sophisticated/higher dimension input features. I also think a distribution matching analysis is needed, to measure the disparate impact of using DP [1].\n[1] Bagdasaryan E, Poursaeed O, Shmatikov V. Di!erential privacy has disparate impact on model accuracy. Advances in neural information processing systems. 2019;32.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper first makes the case for why di!erentially private training of di!usion models could be beneficial, and then proposes a DP-SGD based training method for di!usion models. More specifically, the authors claim that the sequential denoising and the stochastic sampling used in di!usion models make them better candidates for di!erentially private data synthesis, compared to their predecessors, GANs. They also claim that the DM objective function is more robust than the adversarial training one in GAN, which is prone to model collapse. The DP training method that the authors propose introduces noise multiplicity, which reduces the variance of the di!usion objective, helping the overall convergence of the model without adding privacy expenditure. The authors empirically show that DPDM generated synthetic data a) outperforms previous methods in terms of FID b)outperforms previous synthetic methods in terms of downstream classification c) outperforms task-specific DP-SGD trained classifiers.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. I would like to see a discussion of the impact that this DP-generation has on di!erent subgroups of data, and how it compares to GAN generated images. What demographics get more underrepresented? I would honestly guess that distribution representation-wise the proposed DP-DM would be better and more diverse and be less under-representing. but I think results are needed on this.\n2. what limitations do the authors see in terms of scaling up this method, to use it in the wild and on larger images?\n3. I wonder if pre-trained features could be used to improve performance further? or if a pre-trained fine-tuned dp-sgd classifier would outperform this method. either way it wouldn't diminish the\n4. I wonder if the authors have tried to find the optimal number of generated samples needed for classification (downstream task), as in what is the minimum number of generations to get a certain level of accuracy, and how it changes with different $\\epsilon$ levels? as in can we compensate tighter guarantees by generating more samples? ", "broader_impact_concerns": "see above. ", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "xEs9HV9V59", "reviewer": "Reviewer_U9or", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n* The noise multiplicity is a clever idea that boosts utility without additional privacy costs.\n* It is the first paper that demonstrates the feasibility and promise of DP diffusion models. \n\nWeaknesses\n\n* Some claims are not accurate enough. In \"Claims And Evidence\" I answer \"yes\" as these are mostly easy-to-fix writing issues. But please consider revising them. Please see \"Requested Changes\" for more details.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes to train differentially private image generation models with DP-SGD (DPDM). In order to reduce the variance of the gradients (which is important in DP-SGD training), the paper proposes noise multiplicity which computes losses from *multiple* noisy samples for each real sample. The experiments demonstrate that DPDM achieves good trade-offs bewteen privacy and utility (in terms of sample quality (FID) and downstream classification accuracy).\n", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "# Inaccurate claims\n\n* Section 3.1 says \"Note, however, that the L 2-norm of the noise added in the DP-SGD updates scales linearly with the number of parameters, and therefore smaller networks are generally preferred.\". Section 3.2 has similar claims. It is not necessarily true according to the latest papers. Please see [1][2].\n\n* Section 3.2 says \"Blindly reducing the added noise or increasing the clipping constant C could be fatal, as it decreases the number of training iterations allowed within a certain (ε, δ)-DP budget.\". However, increasing the clipping constant C does not change the privacy cost at all (when other hyper-parameters are kept the same), and therefore, it does not \"decrease the number of training iterations allowed within a certain (ε, δ)-DP budget.\"\n\n* Section 3.2 says \"In theory, this increases the memory footprint by at least O(B); however, in practice the peak memory requirement is O(B^2) compared to non-private training (Yousefpour et al., 2021).\". It is outdated. See techniques such as ghost clipping [3].\n\n* ”Diffusion model config.“ paragraph in Section 3.2 explains why denoising steps at large \\sigma are important. However, steps with small \\sigma are also important as they capture fine-grained image details. We can use the same reasoning in Section 3.2 to argue that we need to put more weights for small \\sigma. Therefore, I do not think the discussion explains sufficiently why putting more weights for large \\sigma is beneficial.\n\n* In \"To the best of our knowledge, it has not been demonstrated before in the DP generative modeling literature that (image) data generated by DP generative models can be used to train discriminative models on-par with directly DP-SGD-trained task-specific models.\", it is better to drop the brackets around images, as it has already been demonstrated in DP text [5].\n\n# Experiments\n\n* It would be great to show how noise multiplicity affects the training time.\n\n* Both noise multiplicity and augmentation multiplicity are beneficial for the same reason (reducing gradient variance). I understand that they can be used in conjunction, but it would still be beneficial to compare them experimentally in case people want to choose one of them given resource constraints. \n\n* In Table 2, how many generated samples are used to train the classifier?\n\n* Section 5.2: Does \"stochastic DDIM\" mean [6] with \\eta>0? If so, which \\eta value do you use here?\n\n# Other minor questions\n\n* For the privacy accountant, the authors can consider the newer numerical accountant [4] which gives tighter bounds on \\epsilon and \\delta than the ones cited in the paper.\n\n* Page 11: The use of Churn sampler is already mentioned on page 9, so we can skip the two places of the \"Churn sampler\" to reduce duplication.\n\n* I want to mention the latest work on generating differentially private data with diffusion models [7,8]. I understand that they are too recent so it is not required to know them, but I just want to bring them to the authors' attention. In particular, [7] shows the value of noise multiplicity.\n\n* Page 2: 'experiments on toy data' is too vague. I do not understand what toy data refers to until page 5. It is better to add a reference.\n\n[1] Li, Xuechen, et al. \"When Does Differentially Private Learning Not Suffer in High Dimensions?.\" Advances in Neural Information Processing Systems 35 (2022): 28616-28630.\n\n[2] Anil, Rohan, et al. \"Large-scale differentially private BERT.\" arXiv preprint arXiv:2108.01624 (2021).\n\n[3] Bu, Zhiqi, Jialin Mao, and Shiyun Xu. \"Scalable and efficient training of large convolutional neural networks with differential privacy.\" arXiv preprint arXiv:2205.10683 (2022).\n\n[4] Gopi, Sivakanth, Yin Tat Lee, and Lukas Wutschitz. \"Numerical composition of differential privacy.\" Advances in Neural Information Processing Systems 34 (2021): 11631-11642.\n\n[5] Yue, Xiang, et al. \"Synthetic text generation with differential privacy: A simple and practical recipe.\" arXiv preprint arXiv:2210.14348 (2022).\n\n[6] Song, Jiaming, Chenlin Meng, and Stefano Ermon. \"Denoising diffusion implicit models.\" arXiv preprint arXiv:2010.02502 (2020).\n\n[7] Ghalebikesabi, Sahra, et al. \"Differentially Private Diffusion Models Generate Useful Synthetic Images.\" arXiv preprint arXiv:2302.13861 (2023).\n\n[8] Lin, Zinan, et al. \"Differentially Private Synthetic Data via Foundation Model APIs 1: Images.\" arXiv preprint arXiv:2305.15560 (2023).\n", "broader_impact_concerns": "No concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "4c3tCZkbFA", "reviewer": "Reviewer_n8Pd", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**: This paper addresses a relevant problem in the field. It addresses the instance-specific problem of training diffusion models with DP. Although the underlying technique, DP-SGD, is simple and has been widely applied, the privacy-utility trade-off requires domain specific knowledge and experimentation. The authors present state of the art results and all experimental details are provided with interesting discussion that can provide guidance when implementing DMs with privacy. \n\n**Weaknesses**: One of the main challenges of differential privacy are the computational and memory resources it requires mostly due to per-example clipping. Authors briefly mention the memory cost in section 3.2. But I think a more in depth discussion and a comparison with other techniques is necessary. The appendix summarizes resources but a more clear comparison with other methods would be interesting to have. \n\nAlong this lines there is work on efficient clipping (or ghost clipping techniques https://arxiv.org/pdf/1510.01799.pdf which I believe Opacus library uses) that should also be mentioned in this discussion. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This is an experimental paper that addresses the problem of training diffusion models with differential privacy. The main technique is DP-SGD for privatizing the gradients. Although this is not a novel technique, applying it in practical settings requires extensive hyperparameter tuning and instance specific modifications. The authors widely explore different techniques that are useful for diffusion models. \n\nSpecifically, \n\n(1) They introduce the notion of “noise multiplicity” to reduce the variance of the objective function and boost performance. \n\n(2) They explore different model architectures since the private setting adds noise with magnitude that increases with the number of parameters. \n\n(3) They explore different configurations for the diffusion model (noise distribution) in the private setting. \nThe authors explore all these techniques in the MNIST and FashionMNIST datasets and compare them with state-of-the-art private models. \n", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- See weaknesses above but mainly include a section discussing time/memory complexity of training models with DP. \n- Given the sensitive topic addressed by the paper it would be a good idea to open source the code. ", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: All reviewers have recommended acceptance, and the authors have already updated their paper addressing the points made by the reviewers.
Decision: Accept (poster)