Don’t Generate Me: Training Differentially Private Generative Models with Sinkhorn Divergence
Authors: Tianshi Cao
Alex Bie
Arash Vahdat
Sanja Fidler
Karsten Kreis
Venue: NeurIPS 2021 Poster (2021)
Keywords: Differential Privacy
Generative Learning
Optimal Transport
Sinkhorn Divergence

Abstract: Although machine learning models trained on massive data have led to breakthroughs in several areas, their deployment in privacy-sensitive domains remains limited due to restricted access to data. Generative models trained with privacy constraints on private data can sidestep this challenge, providing indirect access to private data instead. We propose DP-Sinkhorn, a novel optimal transport-based generative method for learning data distributions from private data with differential privacy. DP-Sinkhorn minimizes the Sinkhorn divergence, a computationally efficient approximation to the exact optimal transport distance, between the model and data in a differentially private manner and uses a novel technique for controlling the bias-variance trade-off of gradient estimates. Unlike existing approaches for training differentially private generative models, which are mostly based on generative adversarial networks, we do not rely on adversarial objectives, which are notoriously difficult to optimize, especially in the presence of noise imposed by privacy constraints. Hence, DP-Sinkhorn is easy to train and deploy. Experimentally, we improve upon the state-of-the-art on multiple image modeling benchmarks and show differentially private synthesis of informative RGB images. 

Don't Generate Me:
Training Differentially Private Generative Models with Sinkhorn Divergence
Tianshi C,,
Alex Sie
Arash Vahdat
Sanja Fidler
Karsten Kreis*
atXiv:submi//4074455 [cs.LG] 1 Nov 2021
'University of Toronto
Vector Institute
University of Waterloo
"NVIDIA
timmahtcnnvidaa.com
yabieBuxaterloo.ca
(avahdat, afidler. kkreis ) dnvidia.com
Abstract
Although machine learning models trained on massive data have led to break- throughs in several areas, their deployment in -rivace-sensitive domains remains limited due to restricted access to data. Generative models trained with privacy constraints on private data can sidestep this challenge, providing indirect access to private data instead. We propose DP-Sinkhom, a novel optimal transport- transport-based generative method for learning data distributions from private data with differen- tial privacy. DP-Sinkhorn minimizes the Sinkhorn divergence, a computationally efficient approximation to the exact optimal transport distance, between the model and data in a differentially private manner and uses a novel technique for control- ling the Slas-variance trade-off of gradient estimates. Unlike existing approaches for training differentially private generative models, which are mostly based on generative adversarial networks, do not rely adversarial objectives, which are notoriously difficult to optimize, especially in the presence of noise imposed by privacy constraints. Hence, DP-Sinkhorn is easy to train and deploy. Ex- perimentally, we improve upon the state-of-the-an on multiple image modeling benchmarks and show differentially private synthesis of informative RGB images. Project page: https: //tp:////vv- http///vvtllllla wallaab.ggthte github. ho/DP-/ DP-Sinkhort
1 Introduction
Modern machine learning (ML) algorithms and their practical applications (e.g. recommender sys- tems [1], personalized medicine [2], face recognition [3), speech synthesis [4], etc.) have become increasingly data hungry and the use of personal data is often a necessity. Consequently, the impor- tance of privacy protection has become apparent both the public and academia.
Differential privacy (DP) is a rigorous definition of privacy that quantifies the amount of information leaked by user, participating a data release [5, 6]. The degree of privacy protection is represented by the privacy budget. DP was originally designed for answering queries to statistical databases. In a typical setting. data analyst (party wanting a data curator (party in charge of safekeeping the database; e.g. a hospital), who makes the query on the database and replies with semi-random answer that preserves privacy. Responding each new query incurs privacy cost. If the analyst has multiple queries, the curator must subdivide the privacy budget spend each query. Once the budget is depleted, the curator can no longer respond to queries, preventing the analyst from performing new, unanticipated tasks with the database.
data; e.g. healthcare company) sends query to
Generative models can be applied general and flexible data-sharing medium [7, 8], sidestepping the above problems. In this scenario, the curator first encodes private data into a generative model;
Work done during internship at NVIDIA.
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

then, the model is shared with the analyst, who can use it to synthesize similar yet different data from the training data. This data can be used in any way desired, such as for data analysis or train specific MI models. Unanticipated novel tasks can be accommodated without repeatedly interacting with the curator, since the analyst can easily generate additional synthetic data as required.
Furthermore, it has been observed that generative models can reveal critical information about their training data [9, 10]. For example, Webster et al. [9] found that modern GANs trained on images of faces produce examples that greatly resemble their training data, thereby leaking private informa- tion. Hence, the generative model must be learnt with privacy constraints to protect the privacy of individuals contributing to the database.
Differentially private learning of generative models has been studied mostly using generative adver- sarial networks (GANs) 17, 11, 12, 13, 14]. While GANs in the non-private setting can synthesize complex data such high definition images [15, 16], their application in the private setting is chal- lenging. This is in part because GANS suffer from training instabilities [17, 18), which can be exacerbated by adding noise to the GAN's gradients during training, common technique im- plement DP. Hence, GANs typically require careful hyperparameter tuning. This goes against the principle of privacy, where repeated access to data need to be avoided [19].
In this paper, we propose Sinkkoon,, a novel method to train differentially private generative models using semi-debiased Sinkhorn loss. DP-Sinkhom is based on the framework of optimal transport (OT), where the problem of learning a generative model is framed as minimizing the op- timal transport distance, a type of Wasserstein distance, between the generator-induced distribution and the real data distribution [20, 21J. DP-Sinkhorn approximates the exact OT distance in the pri- mal space using the Sinkhorn iteration method [22]. Furthermore, we propose novel semi-debiased Sinkhorn loss to optimally control the bias-variance trade-off when estimating gradients of this OT distance in the privacy preserving setting. Since our approach does not rely on adversarial compo- nents, it avoids any training instabilities and removes the need for early stopping (stopping before catastrophic divergence of GANs, as done, for example, in [15]). This makes our method easy to train and deploy in practice. To the best of our knowledge, DP- Sinkhor is the first fully OT-based approach for differentially private generative modeling.
In summary, we make the following contributions: (i) We propose DP-Sinkhom, a flexible and robust optimal transport-based framework for training differentially private generative models. (ii) We demonstrate a novel technique to finely control the bias-variance trade-off of gradient estimates when using the Sinkhorn loss. (iii) Benefiting from these technical innovations, we achieve state- of-the-art performance on widely used image modeling benchmarks for varying privacy budgets, both in terms of image quality (as measured by FID) and downstream image classification accuracy. Finally, we present informative RGB images generated under strict differential privacy without the use of public data, with image quality surpassing that of concurrent works.
2 Related Works
The task of learning generative models on private data has been tackled by many prior works. The general approach is to introduce privacy-calibrater noise into the model parameter gradients during training. A long line of works have explored combinations of GANs and differential privacy. DP- GAN [7] first introduced the idea of combining differential privacy with GANs in simple scheme where DPSGD [23] is used when updating the generator. This is followed up a year later by dp- GAN [11], which adds decaying clipping threshold that heuristically matches the decreasing gra- dient magnitude during training. DP-CGAN [24] adds class conditioning to DPGAN for generation of conditional data. PATE-GAN [25] adopts the PATE framework for generative learning by using PATE [26] to train private student discriminator; only generated images are scored by this student discriminator to train the generator. This work is improved by G-PATE [27], which uses random projections and gradient quantization to directly aggregate discriminator gradients for updating the generator. Importantly, G-PATE makes the point that only the generator is released in the DP gen- erative learning task, thus large (~~000ss ensemble of non-private discriminators can be used to train a private generator. GS- GS-WGAN [13] brings this idea back to DPSGD- -based training, in which the gradient aggregate from an ensemble of discriminators is processed by the Gaussian mechanism. Unlike DP-GAN, this performed on the image gradient, which has fewer dimensions than the parameter gradient. Datalens [1 [14] further improves upon G-PATE by introducing TopAgg three

---

step gradient compression and aggregation algorithm, which provides stable, quantized discrimina- tor gradients low privacy cost.
is well documented that GANs are unstable during training [17, 18] due to the non-optimality of the discriminator producing large biases in the generator gradient [20]. This problem is critical in the context of DP, where the imposed gradient noise can increase training instabilities and interaction with private data (for example during hyperparameter tuning) should be limited. Our approach cir- cumvents these issues by not relying on adversarial learning schemes. Furthermore, state-of-the-art methods [13, 14] rely on training large number of discriminators take advantage of the subsam- pling property of differential privacy. This hinders their practical usefulness as the discriminators require large amounts of GPU/TPU memory. In contrast, only a single generator network trained in Sp-Sinnhorn, making our approach more amenable to various hardware configurations.
Other generative models have also been studied in the DP setting. [28] partitions the private data in clusters and learns separate likelihood-based models for each cluster. [29] uses Maximum Mean Discrepancy with random Fourier features. While these works do not face the same stability issues as GANs, their restricted modeling capacity results in these methods mostly learning prototypes for each class. DP-Sinkhorn is better at using the modeling capacity of neural networks produce high utility synthetic data while preserving privacy. Lastly, while [30] produced strong empirical results, their privacy analysis relies on the use of Wishart noise on sample covariance matrices, which has been proven to leak privacy [31]. Hence, their privacy protection is invalid in its current form.
3 Background
3.1 Notations and Setting
Let X' denote a sample space, P(x)) all possible measures on X, and 2 C Rd the latent space. We are interested in training generative model g + X' such that its induced distribution μ = g with noise source & E P(2)) is similar to observed v through an independently sampled finite sized set of observations D {y} In our case, g is trainable parametric function with parameters A.
3.2 Generative Learning with Optimal Transport
Optimal Transport-based generative learning considers minimizing variants of the Wasserstein dis- tance between real and generated distributions [20, 21J. Two key advantages of the Wasserstein distance over standard GANs, which optimize the Jensen-Shannon divergence [32], are its definite- ness on distributions with non-overlapping supports, and its weak metrization of probability spaces [33]. This prevents collapse during training caused by discriminators that overfit training data.
The OT framework can be formulated in either the primal or dual formulation. In WGAN and vari- ants [33, 34, 35], the dual potential is approximated by an adversarially trained discriminator. These methods still encounter instabilities during training, since the non-optimality of the discriminator can produce arbitrarily large biases in the generator gradient [20]. The primal formulation involves solving for the optimal transport plan- aa joint distribution over the real and generated sample spaces. The distance between the two distributions is then measured the expectation of point-wise cost function between pairs of samples as distributed according to the transport plan.
In general, finding the optimal transport plan is difficult optimization problem. The entropy- regularized Wasserstein distance (ERWD) imposes a strongly convex regularization term on the Wasserstein distance, making the OT problem between finite samples solvable in linear time [36]. Given a positive cost function c: X' X R+ and X 2 O, the ERWD is defined as:
flos W))(mn)> min / c(x, (x) y) + > / log (du(x)du(y) dx,
dr dx(xx))
where II - (x( P(X IfI (xx -)xx v,
= 1).
The Sinkhorn divergence uses auto-correlation terms to reduce the entropic bias introduced by ERWD with respect to the exact Wasserstein distance, canceling it out completely for μ (i.e. (,, = for matching μ = v). This results faithful matching between the generator and real distributions. Here, the Sinkhom divergence as defined in [37].
3

---

Spht
Matix
Wa mn Xhhiiinll
Nen-seniitie,
Cas
J0.89
pass
Dadyy
Sersrive.
X
Croweseer Loss
poss
Xibel
Nass
Cis
10.00
(a(Xi Y)
Seralove,
Laini
pas
Caneratvve Maid
Privai;
backsani pacs
Figure 1: Flow diagram of DP-Sinkhorn for a single training iteration: Batch of generated data is split for the calculation of the cross term and the self term losses. Element-wise differences are captured in the cost matrix. Then, the losses are calculated using the Sinkhorn algorithm. In the backward pass, we impose a privacy barrier behind the generator by clipping and adding noise to the gradients at the generated image level, similar to [13].
Definition 3.1. (Sinkhorn Divergence) The Sinkhorn divergence between measures μ and is de- fined as:
Se)(p, 2W, 2Wx((n)) Wex(1, Wa,(u,
3.3 Differential Privacy
The current gold standard for measuring the privacy risk of data-releasing programs is the notion of differential privacy (DP) [5]. Informally, DP measures to what degree program's 's output can deviate between adjacent input datasets d and d' sets differing by one entry. For a user contributing their data, this translates to guarantee on how much an adversary could learn about them from observing program's output. Here, we are learning a generative model of images, while conditioning on available semantic labels. Hence, we are interested in the domain of image-and-label datasets where each image and its label constitute an entry.
A well-studied formulation of privacy, which allows tight composition of multiple queries and can be easily converted standard definitions DP, is provided by Rényi Differential Privacy (RDP) [38]:
Definition 3.2. (Rényi Differential Privacy) randomized mechanism M D - R. with domain 'D and range R satisfies (a., RRDDP for any adjacent d, d' E D:
(.(Mdd))Md') S,
(3)
where Da is the Rényi divergence of order a. Also, any M that satisfies (a, RDDP also satisfies (c log 1/8 5)-DP
Here, M is a DP-learning algorithm, d training set, and MMd) a generator trained on d. The ran- domized mechanism can often be dissected into deterministic function of the dataset Q D R, the query, and noise injecting random function M : R + R. the privacy mechanism, such that M(d) = M' (d). The sensitivity of a query S(Q) is a property that represents the maximum magnitude of change between outputs of the query when applied to adjacent datasets. For query output ((d) with sensitivity S(Q) and standard deviation of Gaussian noise σ,, the Gaussian + z, the dataset into batches also improves privacy. The effect of subsampling on the Gaussian mecha- nism under RDP has been studied in [39, 40, 41]. Furthermore, privacy analysis of a gradient- based learning algorithm entails accounting for the privacy cost of single queries (possibly with subsam- pling), summing up the privacy cost across all queries (i.e. training iterations in our case), and then choosing the best a. A more thorough discussion of DP can be found the Appendix.
mechanism (g)
(0, o²) satisfies (a, asS (2)))/((22 /2(22))) RDDP [38]. Subsampling
4 Differentially Private Sinkhorn
We propose DP-Sinkhorn (Fig. 1), an OT-based method to learn differentially private generative models that avoids the training instability issues of GANs. In this section, we first provide an overview of DP-Sinkhorn, followed by our novel semi-debiased Sinkhorn loss function. We then analyze the privacy protection of DP- OinShnkor,,, and discuss some design considerations.
4.1 Overview of DP-Sinkhorn

---

Algorithm 1 DP-Sinkhorn
DP-Sinkhorn aims to stably and robustly train generative models on high dimensional data (e.g. images) while preserving the privacy of training data. As discussed in Sec. 2. current state-of-the-art methods in privacy- preserving data generation are reliant on ad- versarial training schemes that are not ro- bust, unstable, and complicated to train. DP-Sinkhorn leverages advancements in OT- based generative learning to do away with the adversarial training scheme. Specifi- cally, training a generative model with DP- Sinkhorn is straightforward end-to-end iter- ative loss minimization process. In each iter- ation, data produced by the generator split according to the debiasing ratio into a "cross" group and "debiasing" group. Empirical OT distances are calculated between the "cross" group and the real data, and between the "de- biasing" group and the "cross group". Gra- dients of the OT distances with respect the generated data are calculated and backpropa- gated to the generator. Privacy protection enforced by clipping and adding noise to the gradients of the "cross" group during back- propagation.
L is number of categories. X' is sample space, M is size of private data set. backprop is a reverse mode suto-differentiation function that takes "out'. 'in' and 'grad weights' as input and computes the Jacobian vector product J/n (out) grad weights. Poisson Sample and W', (via Sinkhorn iterations) are defined in Appendix.
Input: private data set d = [(y.1) E X' x {0,... L}}, sampling ratio g. noise scale a, clip- ping coefficient A, generator learning rate a, en- tropy regularization X, debiasing resample fraction p. total steps T.
Output:
* M, n' = floor p)
for t = 1 to do
Sample Y +- Poisson Sample(d, g).
z - (zi) (z:) (atto)) iid
~ Unif(0, 1)
La t (1) (ntn') sad Unif(O, .... L)
X {x+ (x)
gradx x + (X,
xtadx + clip(gradxlomi 1)+
grad giadx/mm==x] <-
/hamt A)
grad, f backprop(X. O, gradx)
- - a * Adam(gradd
end for
4.2 Estimating Sinkhorn Divergence with Semi-Debiased Sinkhorn Loss
Sinkhorn divergence, as expressed in Eq. 2. involves integration over the sample space. Empirical estimation of Eq. 2 based on finite samples is required to train generative model through gradient- based optimization. A solution suggested by previous works [37, 21] would be to replace p and with empirical samples from each distribution.
Definition 4.1. The empirical Sinkhorn loss computed over a batch of n generated examples X and m real examples Y is defined as [37]: Sxx Y) 22x WX(X - WY,,YY,
(4)
where WA = CAB @ PÂB. CAB E R with Ci. = c(x;, (x;, are rows of A, B) is the cost matrix between A and B, and PAB is the approximate optimal transport plan that empirically minimizes WxAA (A, computed by the Sinkhorn algorithm.
However, [42] showed that the gradients of (X, Y) are biased estimates of the gradients of Sc,AHH computed over the population. Instead, they proposed a loss formulation that produces unbiased gradients using additional independently drawn samples:
Definition 4.2. Following the notations of Def. 4.1, let X' and Y' denote second batch of gener- ated and real examples. The debiased Sinkhorn loss is then defined as [42]:
Sax (X, Y, x', Y') (X. Wx - WY, Y').
(5)
In comparison with Def. 4.1, Def. 4.2 comes with higher variance (only W3 (X, Y) contributes to variance in Def. 4.1). Unfortunately, privacy constraints in the DP setting prevent us from using very large batch sizes or very long training periods with low learning rates to effectively reduce variance. Hence, the variance incurred from using the unbiased estimator more difficult to handle in the DP setup. Furthermore, Def. 4.2 draws two batches of real data in every training step, thereby increasing the privacy cost of each step. Nonetheless, Def. 4.2 an unbiased estimator with better convergence properties. We now discuss how we overcome the above issues in DP- Sinkhorn.
2 is the Hadamard product.
5

---

First, we make the observation that the Wa(Y, Y') term does not contribute to gradients of the generator. Hence,
omit it from Se,A- Next, we propose loss formulation that interpolates between biased and unbiased Sinkhorn divergence. As observed in previous works, it can sometimes be beneficial to control blas-variance trade-offs through mixing biased and unbiased gradient esti- mators [43]. Instead of completely resampling the generator for X', we reuse some of the samples in X when computing (X,, This provides better control over the bias-variance trade-off when empirically estimating gradients.
Sinkhorn loss) For a mixture mixturefraactinn p E [0, 1] and natural number n' = floor(n x p). Given + n' generated samples X E xxatn' and real samples Y E Xm the semi-debiased Sinkhorn loss is defined as:
Definition 4.3.
S.p.p X, Y) = [000 Y) - (X (0m), xnnlmn)
where [(a:bl denotes the contiguous rows of X starting from a and ending with b
When p= 1, Eq. 6 is equal to Eq. 5, whereas when = 0, Eq. 6 recovers Eq. 4 (ignoring the terms in Eqs. 4 and 5 that only depend data Y and are irrelevant during training).
Algorithm/ describes how Eq. is used train a generative model, while additionally modifying the gradient by adding noise and clipping to implement the privacy mechanism described below. Training of the generator proceeds by computing the gradient of the semi-debiased Sinkhorn loss with respect to X. Please also see the Appendix for more details.
4.3 Privacy Protection
Information about real data enters the generator through loss function gradients with respect the generated images. Let G Vx (X, (X, Y) denote the gradients of the semi-debiased Sinkhorn
loss, and let G !labb) denote the contiguous rows of G from a to b inclusive. We modify G as:
A
G = G [0mm] min
| l2" 1); G' G G[mn+n))....
min
G
G concat G + 7, G'), where N(0, A A²0²) concat is applied to the first axis
We observe that V xinmtay Wx WX(X (0m), Y) 0, i.e. G [n=n+t>]] contains information about Y. As such, noise does not need to be added to this term, but we apply clipping to G[an+n]] to preserve the scale between the magnitudes of the gradients. addition, the sensitivity of G is y ||G(Y) G(Y') 2A. The following theorem states the privacy guarantee of DP- Sinkhorn's 's gradient updates, with proofs in the appendix:
Theorem 4.1. For clipping constant A and noise vector y (0, (70") releasing G satisfies (a, -RDP.)
We use the RDP accountant with Poisson subsampling proposed in [41] for privacy composition across updates. Note that the batch size of X is kept fixed, while the batch size of Y follows a binomial distribution due to Poisson subsampling.
4.4 Design Considerations
Advantages of primal form OT When compared to WGAN [33], learning with primal form OT (such as Sinkhorn divergence) has distinct differences. While both are approximations to the exact Wasserstein distance, the source of the approximation error differs. WGAN's source error lies in the sub-optimality of the dual potential function. Since this potential function is parameterized by an adversarially trained deep neural network, enjoys neither convergence guarantees nor feasibility guarantees. Furthermore, the adversarial training scheme can cause the discriminator and generator to change abruptly every iteration to counter the strategy of the other player from the previous iteration [44], resulting in non-convergence. These challenges are exacerbated in the DP setting. In contrast, the suboptimality of the transport plan when computing Sinkhorn divergence can be controlled by using enough iterations, and the bias introduced by entropic regularization can be controlled by using small À values. Training with the Sinkhorn divergence does not involve any adversarial training, converges more stably, and reaps the benefits of OT metrics at covering modes.

---

Table 1: Comparison of DP image generation results MNIST and Pashion-MNIST (c, (10, 10-9)-DP. Results for other methods (G-PATE [27], DP-MERF AE [29], DP-CGAN [24], GS- WGAN [13] are from [13], except Datalens [14]. Results are averaged over runs of synthetic dataset generation and classifier training.
MNIST
Fashion-MNIST
Method
DP-c
Acc (%)
Acc (%)
FID
FID
Log Reg
MLP
CNN
Log Reg
MLP
CNN
Real data
00
1.6
92.2
97.5
99.3
2.5
84.5
88.2
90.8
Nou-priv Sinkhorn (m = 1)
00
54.2
89.0
89.0
91.0
65,8
78.4
79.1
73,9
Non-priv Sinkhorn (m =
no
43.4
87.7
87.3
90.6
63.8
78.4
78.4
73.3
G-PATE
10
177.2
25
51/80.9*
205.8
30
50/69.3²
DP-CGAN
10
179.2
63
243.8
50
46
DP-MERF AE
10
161.1
68
213.6
56
62
DataLens GS-WGAN
10
173.5
N/A
N/A
80.66
167.7
N/A
70.61
10
61.3
79
80
131.3
65
DP-Sinkhorn (m = 1)
10
61.2
79.5
80.2
83.2
145.1
73.0
72.8
70.9
DP-Sinkhorn (m = 3)
10
55.56
79.1
79.2
79.1
129.4
70.2
70.2
68.9
Cost function The choice of the element-wise cost c(x. y) influences the type of images produced by the generator. We consider mixture between pixel-wise L, and squared Lg losses. L2 loss has smooth gradients that scale with the difference in pixel value, whereas the gradient of Li loss constant in magnitude for each pixel that differs. Therefore, while L2 loss can quickly rein in outlier pixel values, L1 loss can encourage generated image pixels to closely match those of the real image, promoting sharpness. We define the element wise cost function as Cm (x.y) y) = 9
+ y² L1 (x. y), where y) ||x yll2. L1 (x. y) = xx y and is scalar mixture weight. Class conditioning is also achieved through the cost function by concatenating a one-hot class embedding both the generated images and real images, similar to the approach used in [42]. Intuitively, this works by increasing the cost between image pairs of different classes, hence shifting the weight of the transport plan (Px in Eq. 4) towards class-matched pairs.
5 Experiments
We conduct experiments differentially private conditional image synthesis, since focus is generating nigh-dimensional data with privacy protection. We evaluate our method on both visual quality and data utility for downstream classification tasks. Additional experiments and analyses of the proposed semi-debiased Sinkhorn loss can be found in the Appendix. Code will be released through the project page*.
5.1 Experimental Setup
Datasets We use image datasets: MNIST [45], Fashion-MNIST [46], and CelebA [47] down- sampled 32x32 pixels. For MNIST and Fashion-MNIST, generation conditioned regular class labels; for CelebA we condition on gender.
Metrics In all experiments, we compute metrics against a synthetic dataset of 60k image-label pairs sampled from the model. For a quantitative measure visual quality, we report FID [48]. To measure the utility of generated data, we assess the class prediction accuracy of classifiers trained with synthetic data on the real test sets. We consider logistic regression, MLP, and CNN classifiers.
Architectures & Hyperparameters We implement DP-Sinkhorn with two generator architec- tures. We adopt a four layer, convolutional architecture from DCGAN [49] for MNIST and Fashion- MNIST experiments, and twelve layer residual architecture from BigGAN [15] for CelebA exper- iments. Class conditioning achieved by providing one-hot encoding of the label to the generator, and concatenating the one-hot encoding to images when calculating the element wise cost. We set == for MNIST and Mushion-MNISS experiments, and A=5 for CelebA experiments. Complete implementation details can be found in the Appendix.
'The G-PATE [27] authors report much more accurate classification results than reported in [13]. The visual quality samples both papers roughly the same. 'https:/ /nv- tlabs. gi thtuh.... SL/DP-SSinkhozz
7

---

4
G-PATE
7
DP-CGAN
1
4
3
4
8
0
3
3
4
6
7
9
9
DP-MERF AE
0
3
3
7
6
7
8
9
Datalens
0
3
4
4
9
0
2.
3
4
5
6
8
9
GS-WGAN
0
2
3
4
5
6
8
9
0
2
3
4
5
6
7
8
9
DP-Sinkhorn
0
/
2
3
4
5
6
7
8
9
(m 1)
0
1
3
3
4
5
6
7
8
9
0
.
3
4
5
6
8
DP-Sinkhom
(m 3)
0
/
3
4
5
6
5
2
3
4
5
6
7
8
9
Figure 2: Images generated at (10, 10 DDP for MNIST and Fashion-MNIST by various methods.
Datalens images obtained from [14]; images of other methods obtained from [13].
Figure 3: Images generated CelebA by Datalens (Left) and DP-Sinkhorn (Right). Datalens images obtained from [14].
Privacy Implementation Our models are implemented in PyTorch. We implement the gradi- ent sanitization mechanism by registering a backward hook to the generator output. MNIST and Fashion-MNIST experiments target (10, 10-5, 10-5)-DP while CelebA experiments target (10, 10 10-0-DP. Details are in the Appendix.
5.2 Experimental Results Standard Benchmarks
In Table 1, we compare the performance of two DP-Sinkhorn variants with other methods on MNIST and Mushion-MNIST. We use p=0.2 for the semi-debiased loss, which was determined through grid search. The two variants use different weights (m=1 and m=3) for the L1 loss in the cost function. Given the same privacy budget, DP-Sinkhorn with m=1 generates more informative examples than previous methods, as demonstrated by the higher accuracy achieved by the downstream classifier. On the more visually complex Fashion-MNIL MNIST, DP-Sinkhorn's 's lead especially pronounced, beat- ing previous state-of-the-arn results by significant margin. DP-Sinkhorn with m=3 achieves lower FID than all baselines, while still maintaining downstream accuracy similar to GS-WGAN We hy- pothesize that giving more weight to the L1 loss improves FID because L1 is more sensitive to small differences in pixel values, thereby encouraging sharper edges. Images generated by DP-Sinkhorn are visualized in Fig. 2. DP-Sinkhorn produces more visual diversity within each class compared to the baselines, which likely benefits DP-Sinkhomn's downstream classification performance.
Robustness We evaluate the training stability of DP-Sinkhorn (m 1, p = 0.2) with different learning rates and two optimizers (Adam [50] and SGD) on MNIST. We perform the same parameter sweep on GS- WGAN for comparison as it the strongest baseline we are comparing to. Results illustrated in Fig. 4a. We find that DP-Sinkhorn reliably converges for sufficiently small learning rates, and it is not sensitive to the choice of optimizer. In contrast, GS- AGAN, relying on adversarial training, suffers from non-convergence for learning rates too big to the choice of optimizer. Exact numbers are reported in the Appendix.
small, and is very sensitive
Privacy Utility Trade-off Stronger privacy protection can be attained by training DP-Sinkhorn for fewer iterations at the cost of utility and image quality. We evaluate the performance of DP- Sinkhorn at various privacy budgets and contrast it to GS-WGAN (Fig. 4b). DP-Sinkhorn shows strong performance among wide range of privacy budgets, and provides good downstream utility
http:://ggttttt..oooo////aaa//// (MIT License) [13]

---

part
OF Babie Ape
(a) Comparing hyperparameter sen- (b) FID and utility of DP-Shnnhom (c) Bias-variance trade-off of sitivity of DP-Sinkhorn WGAN on MNIST. Error rate is at various on MNIST.
GS- (m=1 and m=3), and GS- SG-N the gradient estimator over
semi-debiasing parameter p.
calculated as 1 Accuracy. Figure 4: Analyzing hyperparameter choices in DP-Sinkhorn.
even at small privacy budget of c 2.33, significantly outperforming GS- WGAN. Note that we found GS-WGAR to require significantly more memory than DP-Sinkhorn, since it uses multiple discriminators for different parts of the data. In our experiments, DP-Sinkhorn can fit comfortably on an 11GB GPU, while GS-WGAN requires 24GB of GPU memory. Hence, DP-Sinkhorn arguably more scalable to very large datsets.
Table 2: Ablating loss functions, debias- ing, and gradient perturbation mechanism on MNIST.
Analysis of Semi-debiased Sinkhorn Loss To study why our novel semi-debiased Sinkhorn loss outperforms both fully-dehiased and fully-biased Sinkhorn losses, we evaluate bias and variance of the semi-debiased Sinkhorn loss-based gradient estimator Gp = (x)(((()), Y). We sample gen- erator gradients with respect the semi-debiased Sinkhorn loss with different p and plot bias and variance (Fig. 4c). Each line represents a gen- erator trained with a different p on MNIST. For each value of p, we compute Gp on three hundred batches of real and generated data to obtain its av- erage and sample variance, Since G1 is unbiased, we use it as the ground truth when computing bias. Variances of each model's gradients are normal- ized with respect variance of Gg-
Image Gradient Perturbation
Loss
Debiasing
FID
Acc (%)
MLP
CNN
No
La
No
218.6
79.9
80.7
Yes
La
No
124.3
82.0
80.8
Yes
73.9
68.4
65.7
Yos
Litaa
88.6
76.6
76.1
Yes
Full
98.0
63.0
61.5
Yes
Litha
Semi
61.2
80.2
83.2
Table 3: DP image generation results on down- sampled CelebA. We include results from [14] for context, but note that their experiment uses a 64x64 resolution and a larger 5 of 10-5
Method
(t. 10 ---DP
FID
Acc (8)
MLP
CNN
We observe two prominent trends from this graph. First, as we increase bias decreases and vari- ance increases. This requires us to find balance in the trade-off between bias and variance. Sec- ond, we see flatter curves for generators trained with smaller p. As p affects bias and variance through changing the number of resampled gen- erated images, we can deduce that training with smaller p likely results in greater similarity between generated images, which improves consistency across generated images at the cost of diversity. That is, if the generator is mode collapsed, p would have no effect on the bias-variance trade-off, as re- sampling the latent variables would produce the same images. While previous works [42] found fully-debiased (p = 1) Sinkhorn loss to provide higher performance, we find a small amount of debiasing (p- 0.2) to perform best. Our hypothesis is that because training in a privacy-preserving manner is restrictive in batch size and number of iterations, the increased variance of the fully- debiased loss is more detrimental. In particular, in the DP setting we cannot simply increase batch sizes or train with more iterations and lower learning rates to counteract high loss variances, as this would incur increased privacy costs. In contrast, our novel semi-debiasing provides control over the trade-off between consistent low-variance gradients and less biased objectives. This also demon- strates how training in the DP setting differs from the non-private setting, hence requiring ideas and tailored methods.
Real data Sinkhoen
1.1
91.9
95.0
0c
129.5
80.8
82.2
DP Sinkhom
168.4
76.2
75.8
Dutal Len [14]
(10, 10 9)
320.8
N/A
72.9
Alltions We study the impact of perturbing image vs. parameter gradients, design of element- wise cost function, and debiasing on performance in the MNIST benchmark. We start with the simplest model, using parameter gradient perturbation, T.2 loss and no debiasing, and incrementally add components. We use m=1 when adding J, loss, and p=0.2 when adding semi-debiasing. The

---

clipping bound A is tuned separately for the variant with parameter gradient perturbation, while the other hyperparameters are kept fixed. In Table 2, we see that DP-Sinkhom with parameter gradients is already competitive in downstream accuracy, but has poor FID in comparison to using image gradients. We observe that DP-Sinkhorn with L22 loss yields good downsteam task performance, but has higher FID than the L loss variant. Mixing I.1 and Ta loss strikes a balance between better FID and downstream accuracy, We also observe that using a fully debiased gradient estimator is detrimental to performance, which we postulate is due to its high variance. The semi-debiased variant performs better than both the biased and the debiased variants.
5.3 Experimental Results on CelebA
downsampled CelebA. We evaluate whether DP- Sinnknnhom is able to synthesize RGB images that are informative for downstream classification. Despite its simplicity, DP-Sinkhorn generates informative images for gender classification, seen in Tab. 3 (uninforma- tive images would correspond to a =50% classification ratio). Qualitatively, Fig. 3 shows that DP-Sinkhorn can learn meaningful representations of each semantic class (male and female) and produces some in-class variations, while avoiding details that could uniquely identify individuals. Concurrent to our work, Datalens [14] was also applied to gender-conditioned generation of CelebA images, albeit with a different image resolution than ours. Images generated by DP-Sinkhorn clearly resemble faces, while those generated by Datalens are blurrier. We also attempted train GS- WGAN on CelebA, but couldn't 't obtain meaningful results using the default hyper-paramettee
We also evaluate DP-Sinkhome
Conclusions
We propose DP-Sinkhorn, a novel optimal transport- based differentially private generative model. Our approach minimizes new semi-debiased Sinkhorn loss in a differentially private manner. It does not require any adversarial techniques that are challenging to optimize. Consequently, DP- Sinkhorn is easy to train, which hope will help its adoption in practice. We experimentally demonstrate superior performance compared to the previous state-of-the-ar both in terms of image quality and on standard image classification benchmarks using data generated under DP. Our model is applicable for varying privacy budgets and is capable of synthesizing informative RGB images in a differentially private way without using additional public data. We conclude that robust models such as ours are promising direction for differentially private generative modeling.
Limitations and Future Work Our main experiments only used simple pixel-wise L1- and Lg- losses as cost function, yet achieve better performance than GAN-based methods. This suggests that in the DP setting complexity in model and objective are not necessarily beneficial. Nonetheless, limited image quality is the main challenge in DP generative modeling and future work includes designing more expressive generator networks that can further improve synthesis quality, while sat- isfying differential privacy. To this end, kernel-based cost functions may provide better performance on suitable datasets. Our experiments were performed on widely-used image benchmarks for dif- ferentially private generative learning. Future works may extend our method to other data types and domains. In particular, since privacy is an important consideration for medical data, applying DP-Sinkhorn to medical datasets (such as those used in [51]) could be of high practical interest.
Broader Impact Our work improves the state-of-the-art in privacy-preservin generative model- ing. Such advances promise significant benefits to the machine learning community, by allowing sensitive data to be shared more broadly via srivacy-preserving generative models. We believe the strong performance and robustness of DP-Sinkhorn will facilitate its adoption by practition- ers. Although DP-Sinkhorn provides privacy protection in generative learning, information about individuals cannot be eliminated entirely, as no useful model can be learned under (0, O)-DP. This should be communicated clearly to dataset participants. We recognize that classifiers learned with DP can potentially underperform for minority members within the dataset [52, 53, 54J, which may also be the case for classifiers trained on data produced by DP-Sinkhorn. Addressing these types of imbalances is an active area of research [55, 56, 57, 58].
Acknowledgments and Disclosure of Funding
This work was funded by NVIDIA. Tianshi Cao and Alex Bie acknowledge additional revenue from Vector Scholarships in Artificial Intelligence, which are not in direct support of this work.
10

---

References
[1] C. A. Gomez-Uribe and N. Hunt, "The netflix recommender system: Algorithms, business value, and innovation," ACM Trans. Manage. Inf. Syst. vol. 6, Dec. 2016.
[2] D. Ho, S. R. Quake, E. R. B. McCabe, W. J. Chng, E. K. Chow, X. Ding, B. D. Gelb, G. S. Ginsburg, J. Hassenstab, C.-M. Ho, W. C. Mobley, G. P. Nolan, S. T. Rosen, P. Tan, Y. Yen, and A. Zarrimpar, "Enabling Technologies for Personalized and Precision Medicine," Trends Biotechnol., vol. 38, no. 5, pp. 497-518, 2020.
[3] M. Wang and W. Deng, "Deep Face Recognition: A Survey," arXiv preprint
2020.
[4] A. v. d. Oord, S. Dieleman, H. Zen, Simonyan, O. Vinyals, A. Graves, N. Kalchbrenner, A. Senior, and K. Kavukcuoglu, "Wavenet: A generative model for raw audio," arXiv preprint arXiv: 1609. 0349,, 2016.
[5] C. Dwork, F. McSherry, K. Nissim, and A. Smith, "Calibrating noise to sensitivity in private data analysis." in Theory of cryptography conference, pp. 265-284, Springer, 2006.
[6] C. Dwork and A. Roth, "The Algorithmic Foundations of Differential Privacy," Found. Trends Theor. Comput. Sci., vol. 9, p. 211-407, Aug. 2014.
[7] L.. Xie, K. Lin, S. Wang, F. Wang, and J. Zhou, "Differentially private generative adversarial network," arXiv preprint arXiv: / 1802.6633, 2018.
[8] S. Augenstein, H. B. McMahan. D. Ramage, S. Ramaswamy, P. Kairouz, M. Chen, R. Math- ews, and B. A. Arcas, "Generative Models for Effective ML on Private, Decentralized Datasets," in International Conference on Learning Representations, 2020.
[9] R. Webster, J. Rabin, I.. Simon, and F. Jurie, "This person (probably) exists. identity member- ship attacks against gan generated faces," arXiv preprint arXiv:2107.0600) 2021.
[10] J. Hayes, L. Melis, G. Danezis, and E. De Cristofaro. 'Logan: Membership inference attacks against generative models," in Proceedings Privacy Enhancing Technologies (PoPETs), vol. 2019, pp. 133-152, De Gruyter, 2019.
[11] I.. Frigerio, A. S. de Oliveira, L.. Gomez, and Duverger, "Differentially private generative adversarial networks for time series, continuous, and discrete open data," in IFIP International Conference on ICT Systems Security and Privacy Protection, pp. 151-164, Springer, 2019.
[12] Yoon, Jordon, and M. van der Schaar, "PATE-GAN: Generating synthetic data with differ- ential privacy guarantees," in International Conference on Learning Representations, 2019.
[13] D. Chen, T. Orekondy, and M. Fritz, "GS-WGAN: A Gradient-Sanitizn Approach for Learn- ing Differentially Private Generators," in Advances in Neural Information Processing Systems, 2020.
[14] B. Wang, F. Wu, Y. Long, L. Rimanic, C. Zhang, and B. Li, "Datalens: Scalable privacy pre- serving training via gradient compression and aggregation," arXiv preprint arXiv:2103. 11109, 2021.
[15] A. Brock, Donahue, and K. Simonyan, "Large scale gan training for high fidelity natural image synthesis," in International Conference on Learning Representations, 2019.
[16] T. Karras, S. Laine, M. Aittala, J. Hellsten, J. Lehtinen, and T. Aila, "Analyzing and improving the image quality of stylegan," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8110-81 8199,119992911 2020.
[17] M. Arjovsky and L. Bottou, "Towards Principled Methods for Training Generative Adversarial Networks," in International Conference on Learning Representations, 2017.
[18] L. Mescheder, A. Geiger, and S. Nowozin, "Which training methods for GANs do actually converge?." in International Conference on Machine Learning (J. Dy and A. Krause, eds.), vol. 80 of Proceedings of Machine Learning Research, (Stockholmsmässann Stockholm Swe- den), pp. 1481-3490,, PMLR, 10-15 Jul 2018.
[19] K. Chaudhuri and A. Vinterbo, "A stability- validation procedure for differentially private machine learning." in Advances in Neural Information Processing Systems 26 (C. J. C. Burges, I. Bottou, M. Welling, Z.. Ghahramani, and K. Q. Weinberger, eds.), pp. 2652-2660, Curran Associates, Inc., 2013.
11

---

[20] O. Bousquet, S. Gelly, I. Tolstikhin, C.-J. Simon-Gabrier Gabriel, and B. Schoelkopf. "From opti- mal transport generative modeling: the vegan cookbook," arXiv preprint wrXiv:1705.06642, 2017.
[21] G. Peyré and M. Cuturi, "Computational Optimal Transport," " Foundations and Trends in Ma- chine Learning. vol. 11, no. 5-6, pp. 355-607, 2019.
[22] M. Cuturi, "Sinkhorn distances: Lightspeed computation of optimal transport." in Advances in neural information processing systems. pp. 2292-2300 2013.
[23] M. Abadi, A. Chu, I. Goodfellow, H. B. McMahan, I. Mironov, K. Talwar, and L. Zhang, "Deep learning with differential privacy," in Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communiccations Security, CCS '16, (New York, NY, USA), p. 308-318, Association for Computing Machinery, 2016.
[24] R. Torkzadehmahani, P. Kairouz, and Paten, "Dp-cgan: Differentially private synthetic data and label generation," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp- 0-0, 2019.
[25] J. Jordon, J. Yoon, and M. van der Schaar, "Pate-gan: Generating synthetic data with differen- tial privacy guarantees," in International Conference on Learning Representations, 2018.
[26] N. Papernot, M. Abadi, Úlfar Frlingsson, I. Goodfellow, and K. Talwar, "Semi-suuprrised Knowledge Transfer for Deep Learning from Private Training Data," in International Confer- ence on Learning Representations, 2017.
[27] Y. Long, Lin, Z. Yang, C. A. Gunter, Liu, and B. Li, "Scalable differentially private data generation via private aggregation of teacher ensembles," 2019.
[28] G. Acs, L.. Melis, C. Castelluccia, and E. De Cristofaro, "Differentially private mixture of generative neural networks," " IEEE Transactions on Knowledge and Data Engineering, vol. 31, no. 6, pp. 1109-1121, 2018.
[29] Harder, K. Adamczewski, and M. Park, "Differentially private mean embeddings with random features (dp-merf) for simple & practical synthetic data generation," arXiv preprint arXiv:2002.1160: 2020.
[30] S. Takagi, T. Takahashi, Y. Cao, and M. Yoshikawa, "P3gm: Private high-dimensional data release via privacy preserving phased generative model," arXiv preprint arXiv:2006.12101, 2020.
[31] A. Sarwate, "Retraction for symmetric matrix perturbation for differentially-private principal component analysis," 2017.
[32] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, "Generative adversarial nets," in Advances in Neural Information Processing Systems 27 (Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger, eds.), pp. 2672-2680, Curran Associates, Inc., 2014.
[33] M. Arjovsky. S. Chintala, and L. Bottou, "Wasserstein gan," arXiv preprint arXiv:1701.07877 2017.
[34] I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin, and A. C. Courville, "Improved training of wasserstein gans," in Advances in neural information processing systems, pp. 5767-5777, 2017.
[35] Miyato, Kataoka, M. Koyama, and Yoshida, "Spectral normalization for generative adversarial networks," in International Conference on Learning Representattinss, 2018.
[36] G. Peyré, M. Cuturi, et al., "Computational optimal transport: With applications to data sci- ence," Foundations and Trends@ in Machine Learning, vol. 11. no. 5-6. pp. 355-607, 2019. [37] J. Feydy, T. Séjoumé, F.-X. Vialard, S.-i. Amari, A. Trouvé, and G. Peyré, "Interpolating between optimal transport and mmd using sinkhorn divergences," in The 22nd International Conference Artificial Intelligence and Statistics, pp. 2681-2690, 2019.
[38] I. Mironov, "Rényi differential privacy," in 2017 EEE 30th Computer Security Foundations Symposium (CSF), pp. 263-275, 2017.
[39] Y.-. Wang, B. Balle, and S. P. Kasiviswanathan, "Subsampled rényi differential privacy and analytical moments accountant," The 22nd International Conference on Artificial Intelli- gence and Statistics, pp. 1226-1235, PMLR, 2019.
12

---

[40] B. Balle, G. Barthe, and M. Gaboardi, "Privacy amplification by subsampling: Tight analy- ses via couplings and divergences." in Advances in Neural Information Processing Systems, pp. 6277-6287, 2018.
[41] Y. Zhu and Y.-X. Wang, "Poission subsampled rényi differential privacy." in International Conference on Machine Learning, pp- 7634-7642, 2019.
[42] T. Salimans, H. Zhang, A. Radford, and D. Metaxas, "Improving GANs using optimal trans- port," in International Conference on Learning Representations, 2018.
[43] B. Poole, S. Ozair, A. Van Den Oord, A. Alemi, and G. Tucker, "On variational bounds of mu- tual information." in International Conference on Machine Learning, pp. 5171-5180, PMLR, 2019.
[44] L.. Mescheder, S. Nowozin, and A. Geiger, "The numerics of gans," in Advances in Neural Information Processing Systems, pp. 1825-1835, 2017.
[45] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, btradient-based learning applied to document recognition," Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324 1998.
[46] H. Xiao, K. Rasul, and R. Vollgraf, "Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms," arXiv preprint /rXiv: 07747, 2017.
[47] Z. Liu, P. Luo, X. Wang, and X. Tang, "Deep learning face attributes in the wild," in Proceed- ings of International Conference on Computer Vision (ICCV), December 2015.
[48] M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter, "Gans trained by a two time-scale update rule converge to local nash equilibrium," in Advances in neural information processing systems, pp. 6626-6637, 2017.
[49] A. Radford, I.. Metz, and S. Chintala, "Unsupervised representation learning with deep convo- lutional generative adversarial networks," arXiv preprint arXiv: 1.04444 2015.
[50] D. P. Kingma and J. Ba, "Adam: A method for stochastic optimization," in 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings (Y. Bengio and Y. LeCun, eds.), 2015.
[51] B. K. Beaulieu-Jones, 7. S. Wu, C. Williams, R. Lee, S. P. Bhavnani, J. B. Byrd, and C. S. Greene, "Privacy-preeerving generative deep neural networks support clinical data sharing." Circulation:
Cardiovascular Quality and Outcomes, vol. 12, no. 7, p. e005122, 2019.
[52] R. Cummings, V. Gupta, D. Kimpara, and J. Morgenstern, "On the compatibility of privacy and fairness," in Adjunct Publication of the 27th Conference on User Modeling, Adaptation and Personalization, UMAP' 19 Adjunct, (New York, NY, USA), p. 309-315, Association for Computing Machinery, 2019.
[53] S. Kuppam, R. McKenna, D. Pujol, M. Hay, A. Machanavajjhala, and G. Miklau, "Fair decision making using
data," CoRR, vol. abs/1905.12244, 2019.
[54] S. Agarwal, Orade-Offss between Fairness, Interpretability, and Privacy in Machine Learning. PhD thesis, University of Waterloo, 2020.
[55] A. Grover, J. Song, A. Kapoor, K. Tran, A. Agarwal, E. J. Horvitz, and S. Ermon, "Bias cor- rection of learned generative models using likelihood-free importance weighting," in Advances in Neural Information Processing Systems, 2019.
[56] K. Choi, A. Grover, T. Singh, R. Shu, and S. Ermon, "Fair generative modeling via weak supervision," in Proceedings of the 37th International Conference on Machine Learning, 2020.
[57] N. Yu, K. Li, P. Zhou, J. Malik, L. Davis, and M. Fritz, "Inclusive GAN: improving data and minority coverage in generative models," in Computer Vision ECCV 2020 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXII, 2020.
[58] J. Lee, H. Kim, Y. Hong, and H. W. Chung, "Self-diagnosing gan: Diagnosing underrepre- sented samples in generative adversarial networks." arXiv preprint arXiv:2102.1 12033, 2021.
[59] J. Feydy, Geometric data analysis, beyond convolutions. PhD thesis, ENS, Mar 2020.
13


Reviews:
[{"review_id": "jPw-WlDQU6g", "reviewer": "Reviewer_U7YX", "summary": "The paper presents a DP model based on Sinkhorn GAN. Different from DP GAN models that are based on adversarial loss, they use an optimal transport (OT) distance similar to Wasserstein distance called Sinkhorn divergence. The idea is simple and incremental, it is an extension of Sinkhorn GAN. Experimental results looks promising but some important results are missing.  \n", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "- In Section 2, the authors briefly mentioned some of the existing DP-GAN methods based on DP-SGD or PATE. Many these techniques or new developed version of them are already applied noised on the gradient of the Wasserstein distance and handled instability of traditional GANs. It is not clear how the proposed method is different from the state-of-the-art techniques. Maybe a good comparison between DP-Sinkhorn and DP-WGAN can better address this issue.\n\n- Please explain how DP-Sinkhorn can be applied on non-image data (e.g., tabular data) where both continues and categorical features exist. For GANs, there are several techniques that handle tabular data (e.g., CTGAN).\n\n- Please provide more information about cross group and debiasing group and why noise is only added to the cross group?\n\n- One of the disadvantage of GAN models are the long training time. It would be great if the authors show the training time comparison between the proposed model and DP-GAN models.\n\n- In experimental results, why CelebA data is missing in table 1 (comparison results)? Why for this dataset, the image size is reduced to 32*32? How the model can handle large scale datasets (images with high resolution)? Why Figure 3 does not include other benchmarks? \n\n- Please explain how the proposed model performs on imbalanced datasets and if it suffers from mode collapse similar to some traditional GAN models?", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "It is better if the authors mention few limitations and future directions for this model", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": ["I don’t know"], "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "a7EqVmVlQC", "reviewer": "Reviewer_oTPG", "summary": "The paper considers the problem of generating private synthetic data using generative adversarial networks trained with differential privacy. While this has been considered, the authors propose an improvement by optimizing Sinkhorn divergence, which has been shown to be a successful strategy for nonprivate GANs. Due to challenges which do not arise in nonprivate GAN training (a difficult bias variance tradeoff), they use a weakening of the unbiased Sinkhorn divergence. They evaluate their strategy on multiple datasets, measuring both FID and a classifier's performance on synthetic data.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros:\nThe experimental results are strong (improving over existing DP strategies for synthetic data), although evaluated on small datasets. I'm a bit surprised that the classifier performance on MNIST is so bad, when even at somewhat small epsilon values, just because of how easy MNIST is.\n\nBeing able to avoid noising G' is nice.\n\nThe authors put a solid amount of effort into ablation to justify their strategy.\n\nCons:\nEpsilon is really large - the typical way of justifying this is showing that privacy attacks are not effective - could you do this here? I have seen, for example, membership inference being run on discriminators to test the privacy of generative models.\n\nThe l_1 and l_2 losses are pretty simple for image data; I have seen perceptual distance used in place of these for image generation.\n\nI'd be interested in seeing results on non-ML images, e.g. medical image datasets or medical records. This seems to be fairly standard in the DP synthetic data literature.\n\nOverall, I am happy with the paper. I'm not super familiar with the generative model literature, but it seems to take a known nonprivate technique and apply it, overcoming challenges that pop up.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Limitations and societal impact are fine.", "review_text": "", "extra_scores": {"ethical_concerns": "N/A", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "_KDIInkMvoD", "reviewer": "Reviewer_Xbc7", "summary": "This paper proposes a new technique for generating differentially-private synthetic data. The approach avoids adversarial training altogether, which has historically given poor model accuracy due to its instability and sensitivity to noise. Instead, the authors minimize Sinkhorn divergence, which is a computationally-tractable approximation to the optimal transport distance between the true distribution and the generated one. The authors demonstrate empirical gains over existing techniques for DP generative models. ", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall, I think this is a potentially interesting idea, and I believe there is a lot of value to pursuing non-adversarial techniques for DP generative models. The idea of using sinkhorn divergence to (non-adversarially) train a generative model appears to be new, to the best of my knowledge. \n\nMy main concern is that the proposed generative model simply doesn’t work very well in the non-private setting, as evidenced by its poor FID scores. This seems to suggest that there is no hope of making this approach ever have “good enough” fidelity to compete with non-DP data. That being said, Sinkhorn-DP does seem to have slightly better FID and prediction accuracy than the many competing baselines, so that is an interesting observation. Though the difference is not large compared to, say, GS-WGAN. So when we’re talking about FID scores that in the hundreds, I would have ideally liked to see a more substantial improvement over the state-of-the-art scores. \n\nTechnically, I found the ideas in this paper to be a bit limited. The approach is heuristic, and there is no fundamental reason this approach should work better, other than getting rid of adversarial training. The Sinkhorn-DP approach seems to be very sensitive to hyperparameters as shown in the evaluation, and it is only evaluated on small-scale datasets. The theoretical claims are limited to proving DP of the scheme, which follows trivially from the construction. So in terms of contribution, I am not sure this paper deepens our understanding of how to improve DP generative models. That being said, it does appear to give (modest) gains over SOTA.  \n\nI was not able to understand a few details of the approach from the paper: \n\nDef. 4.1 – how is the cost matrix C_AB defined, and what is its intuitive meaning? (I did see you partially discuss in this in Sec. 4, but it should come earlier, and I still wanted more details on how to interpret C_AB). What is the Sinkhorn Algorithm used to compute P_AB, and why isn’t W_λ(X, X)=0? \n\nAlgorithm 1: Why do you clip grad(X[n:n+n’]) if you are not adding DP noise? What is the meaning of the categories 0,…, L? Are you assuming labelled data and conditional generation? If so, this isn’t described in the problem setup… (Again, I later saw in the evaluation that you are considering conditional generation, but this wasn’t clear upfront)\n\nMinor comments:\nIn contrast, only a single generator network is trained in DP-Sinkhorn, making our approach more amenable to various hardware configurations -> do you mean a single discriminator? The previous lines are about multiple discriminators.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors seem to be aware of several of the limitations of their approach. It may be helpful to discuss the fact that the non-private model has very poor fidelity, but the DP version still outperforms existing baselines. Is this fundamental? I think this is an interesting question to explore. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2.5"}, "extra_text": {}}, {"review_id": "I4fu7V9OIu5", "reviewer": "Reviewer_GTXj", "summary": "The paper proposes DP-Sinkhorn, an optimal-transport based generative model for privacy-preserving data generation based on differential privacy.\n\nOptimal transport based generative models minimize variants of the Wasserstein distance, in particular an entropy-regularized version. The Sinkhorn divergence adds an autocorrelation term which cancels out the entropic bias completely when the distance is zero. [36] showed that the gradients of the empirical Sinkhorn loss are biased and proposes using independently drawn samples from the generator, the \"debiased\" Sinkhorn loss.\n\nThis paper proposes the \"semi-debiased\" Sinkhorn loss which, instead of using independently drawn samples, splits the sample along the batch dimension. \n\nDifferential privacy is added by perturbing gradients with the privacy level determined using the RDP accountant. \n\nThe method of using the semi-biased Sinkhorn loss with DP is termed DP-Sinkhorn and the paper argues that the lack of adversarial training, as opposed to GANs, avoids training instability and the need for early stopping. \n\nExperimental results indicate better performance on MNIST, Fashion-MNIST and Celeb-A compared to other DP generative models. ", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "In general, I found the paper to be confusing in terms of what it is arguing its main contributions are. \n\nThe paper casts itself as a contribution in privacy-preserving generative models, but there is not really any novelty in the DP domain as existing privacy mechanisms are used to achieve DP in the same way they are used for existing DP GANs and related models.\n\nRather, the primary novelty is that the authors have proposed the \"semi-debiased\" Sinkhorn loss, compared to the debiased Sinkhorn loss, but it's not clear what this has to do with privacy - and if this does represent a significant improvement, the paper does not dedicate much theory or experiments in attempting to understand its improvement (why the semi-debiased version is better in a general, non-private context).\n\nThis same confusion extends to the experimental results. While the results appear strong, the only included baselines are other DP generative models so it's unclear why the performance is better (is the underlying model just better in a non-private context so it remains better even after DP is added?). This confusion makes it difficult to judge the novelty and significance. \n\nIt seems there are 3 possibilities: 1) the paper is arguing OT-based generative models are better in general than GANs so they remain better when coupled with standard DP mechanisms; 2) the paper is arguing that the \"semi-debiased\" Sinkhorn loss significantly improves over the debiased Sinkhorn loss and that's what's responsible for the gains (making privacy a secondary thought); 3) the paper is arguing there is something specific about the semd-debiased Sinkhorn related to privacy that causes it to perform better (compared to other Sinkhorn losses) when coupled with DP mechanisms. I'm not sure which the paper is intending to argue and what the primary support would be for the claim. \n\nI also found the claim that DP-Sinkhorn \"avoids the need for early stopping\" strange since this does not appear to be something specific to adversarial learning.\n\nFinally, the writing in section 4.2 should be more clear and explicit since it's describing the main contribution, e.g. X' is never defined. It still remains unclear to me why the performance should be that different by simply splitting the batch rather than sample from the generator twice.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "There is not much discussion of limitations", "review_text": "", "extra_scores": {"ethical_concerns": "N/A", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3-4"}, "extra_text": {}}, {"review_id": "FEZPH76Ynof", "reviewer": "Reviewer_BgbF", "summary": "The paper studies the problem of training generative models under differential privacy constraint. To this end, they propose a new optimal transport based objective, the semi-debiased Sinkhorn loss, analyze its privacy properties, and evaluate its performance on a number of image datasets.\n", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality: The choice of optimizing the semi-debiased Sinkhorn loss is, to the best of my knowledge, novel. The task itself (differentially private generative models) and techniques (gradient clipping +  adding Gaussian noise) are well-known in the literature. While the authors do an adequate job of surveying related work, their exposition as to other methods could be significantly improved in the experiments section (Table 1). It’s not clear exactly how the objectives and training procedures differ between these methods and those proposed by the authors.  A quick summary of how these other algorithms work would be beneficial so as to better understand the contributions of their paper.\n\nQuality: The submission is technically sound. The proofs appear to be correct, although I was not able to verify them in precise detail. The methods used in evaluation seem appropriate, although it would be helpful to the reader if the author’s provided confidence intervals for the results in table 1 to get a better sense of the variance in these methods. \n\nClarity: The paper is for the most part well written. \n\nWhile their methods do appear to provide some improvement on MNIST and CelebA, these datasets are relatively low dimensional and it would be valuable to understand how the procedures introduced in this paper perform in more complicated settings. Given that the theoretical contributions are based on standard applications of DP tools, the overall impact of the work seems to me like it hinges on the quality of the empirical results. To this end, I would have liked to see experiments in a wider variety of domains to be better convinced of the impact that these ideas will have in practice.\n\n\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes an DP algorithm for generative modeling, based on Sinkhorn divergence. This approach is non-adversarial, leading to a simpler and more stable optimization problem that is better suited for making differentially private. On the other hand, empirically, even non-private version of non-adversarial generative models lead to worse models and this limits the possible impact of this work.

Papers at the intersection of two active areas of research are usually hard to evaluate and this paper is no exception. The reviewers were divided on this work. The improvement over previous work is small in FID score, though FID is not a particularly good measure. The generated images are reasonably good.

I find the approach promising and the authors might want to evaluate if this approach leads to more "stable" optimization compare to GAN-based approaches; e.g. is this algorithm less sensitive to the choice of hyperparameters. The authors are also encouraged to evaluate this approach on medical datasets, such as those used in https://doi.org/10.1101/159756

On balance, this paper may lead to a broader range of ideas being explored in this space, and therefore I would recommend acceptance.
Decision: Accept (Poster)