Learning to Noise: Application-Agnostic Data Sharing with Local Differential Privacy
Authors: Alex Mansbridge
Gregory Barbour
Davide Piras
Christopher Frye
Ilya Feige
David Barber
Venue:  (2021)
Keywords: Differential Privacy
Representation Learning
Variational Inference
Generative Modelling

Abstract: In recent years, the collection and sharing of individuals’ private data has become commonplace in many industries. Local differential privacy (LDP) is a rigorous approach which uses a randomized algorithm to preserve privacy even from the database administrator, unlike the more standard central differential privacy. For LDP, when applying noise directly to high-dimensional data, the level of noise required all but entirely destroys data utility. In this paper we introduce a novel, application-agnostic privatization mechanism that leverages representation learning to overcome the prohibitive noise requirements of direct methods, while maintaining the strict guarantees of LDP. We further demonstrate that data privatized with this mechanism can be used to train machine learning algorithms. Applications of this model include private data collection, private novel-class classification, and the augmentation of clean datasets with additional privatized features. We achieve significant gains in performance on downstream classification tasks relative to benchmarks that noise the data directly, which are state-of-the-art in the context of application-agnostic LDP mechanisms for high-dimensional data sharing tasks.

Under review as a conference paper at ICLR 2021
LEARNING TO NOISE: APPLICATION-AGNOSTICC DATA SHARING WITH LOCAL DIFFERENTIAL PRIVACY
Anonymous authors Paper under double-blind review
ABSTRACT
In recent years, the collection and sharing of individuals' private data has become commonplace in many industries. Local differential privacy (LDP) rigorous approach which uses a randomized algorithm to preserve privacy even from the database administrator, unlike the more standard central differential privacy. For LDP, when applying noise directly to high- nigh-dimensional data, the level of noise required all but entirely destroys data utility. In this paper we introduce a novel, application-agnostic privatization mechanism that leverages representation learn- ing to overcome the prohibitive noise requirements of direct methods, while main- taining the strict guarantees of LDP. We further demonstrate that data privatized with this mechanism can be used to train machine learning algorithms. Applica- tions of this model include private data collection, private novel-class classifica- tion, and the augmentation of clean datasets with additional privatized features. We achieve significant gains in performance on downstream classification tasks relative to benchmarks that noise the data directly, which are state-of-the-ee in the context of application-agee aggnosicc LDP mechanisms for high-dimensional data sharing tasks.
INTRODUCTION
The collection of personal data is ubiquitous. and unavoidable for many in everyday life. While this has undeniably improved the quality and user experience of many products and services, evidence of data misuse and data breaches (Sweeney) 1997, Jolly 2020) have brought the concept of data privacy into sharp focus, fueling both regulatory changes as well as shift in personal preferences. The onus has now fallen organizations to determine if they are willing and able to collect personal data under these changing expectations. There is thus a growing need to collect data in a privacy- preserving manner, that can still be used to improve products and services.
Often coined the 'gold standard' of privacy guarantees, central differential privacy (CDP) (Dwork & Roth 2014) protects against adversary determining the presence of a user in dataset. It provides a quantifiable definition, is robust to post- post-processing. and allows for the protection of groups of peo- ple via its composability property (Dwork al., 2006 Dwork & Roth 2014). The CDP framework relies on the addition of noise to the output of statistical queries on a dataset, in such a way that the same information can be extracted from that dataset whether not any given individual present.
One can train machine learning models such that the model is CDP with respect to the training set by using training methods such as DP-SGD (Abadi al. 2016), DP-Adam (Gylberth et al. 2017) or PATE (Paperno et al., 2017). Given access to a clean labelled dataset, one could train, for example, CDP classifier this way. Similarly, one could 'share' data privately by training a generative model, such as a variational autoencoder (VAE) or generative adversarial network (GAN), with a CDP training method, and then construct synthetic dataset satisfying CDP with respect to the training set by generating samples from this model (Xie et al., 2018) Triasteyn & Faltings) 2019; Acs et al.) 2019 Takagi et al. 2021). While some of these approaches require only an unlabelled training set, their applications are limited in that they generate only synthetic samples that are likely under the original training set distribution. Firstly, changes in the data distribution warrant the re-training of the generative model. Secondly, the synthetic points are only representative samples, and so we lack any information about the features of given individuals. Clearly, this limits the range of applications: the model cannot be used to join private and clean datasets, for applications in which

---

Under review as a conference paper at ICLR 2021
expect distributional shift, for privately collecting new data. Furthermore, CDP also requires
a trustworthy database administrator.
Federated learning (McMahan et al. 2017 Agarwal et al. 2018 Rodriguez-Barross et al. 2020)
is a related technique in which a model can be trained privately, without placing trust in a database administrator. The model is trained across multiple decentralized devices, omitting the need to share data with a central server. Unfortunately, by construction, federated learning does not allow for any data to be privately shared collected, and is only concerned with the privacy of the model.
An approach that allows data collection, while protecting the privacy of an individual against even the database administrator, is to construct mechanism that privatizes the features of given in- dividual locally, before collection. Warner (1965) developed mechanism, known as randomized response, preserve the privacy of survey respondents: when answering sensitive (binary) ques- tion, the respondent is granted plausible deniability by giving truthful answer if a fair coin flip returns heads, and answering yes otherwise.
Recent work has further developed this idea, often referred to as local differential privacy (LDP) (Kasiviswanathan al. 2008; Duchi et al. (013). LDP provides a pathemmmticclllyproabbbe privacy guarantee for members of a database against both adversaries and database administrators. Many existing LDP mechanism focus on the collection of low-dimensional data like summary statistics, or mechanisms which do not easily generalize to different data types. Erlingsson et al. (2014), Ding et al. (2017) and Tang et al. (2017) introduce methods for collecting such data repeatedly over time. EElingsson et al. (2014) find that for one time collection, directly noising data (after hashing to a bloom filter) is their best approach for inducing privacy. Ren et al. (2018) extend this bloom filter based approach, and attempt estimate the clean distribution of their LDP data in order to generate a LDP synthetic dataset, though we note that the range of applications is limited, as with CDP synthetic generation above.
In this paper, we adapt well-established techniques from representation learning to address the fun- damental limitation of LDP in the context of hhihh-dimensional data: datapoints in high dimensional space require prohibitive levels of noise to locally privatize (the privacy budget, e, naively scales linearly with the dimensionality). To motivate our approach, consider the fact that it is often good approximation to assume that given nigh-dimensional dataset lives on a much lower dimensional manifold. Applying a general privatization mechanism to low-dimensional representations should thus enable us to learn how to add noise to the high-dimensional data efficiently and application- agnostically. Our approach is inspired by the VAE (Kingma & Welling, 2014; Rezende et al.) [0144); demonstrate that sampling in latent space is equivalent to passing datapoint through an LDP Laplace mechanism. Furthermore, reconstructing a datapoint is equivalent to adding a complex noise distribution to the raw features, thereby inducing LDP.
Our randomized algorithm, which we refer to as the variational Laplace mechanism (VLM), satisfies the strict guarantees of LDP, and is agnostic to both data type and downstream task. We demon- strate that we can use the data privatized with our mechanism to train downstream machine learning models that act on both clean and privatized data at inference time. Furthermore, we demonstrate multiple concrete applications of our model: we privately collect data from individuals for down- stream model training; we use a wunsfer-learinng-inpiree. approach to privately collect data of an unseen class type upon which we train a classifier; and we augment a clean dataset with additional privatized features to improve the accuracy of a classifier on the combined data. None of these appli- cations can be solved with CDP, and we find significant performance gains over the naive approach of directly noising the data.
BASIC DEFINITIONS AND NOTATION
To formalize the concept of differential privacy, we first introduce some definitions and notation.
Definition ((c, d') -central differential privacy): Let A: D + Z be randomized algorithm, that takes as input datasets from the dataset domain D. We say A is (c, -)-eenaa differentially private for 5 2 0, for all subsets S C Z, and for all neighboring datasets D, D' E DD we have
p(A(D) E S) < exp(c) p(A(D)) E S) +
(1)
where for D and D' be neighhoring means that they are identical in all but one datapoint.
2

---

Under review as a conference paper at ICLR 2021
Intuitively, this states that cannot tell (with level of certainty determined by (c, whether an individual is present a database or not. When we say A satisfies e-CDP.
Definition (61 sensitivity): The l sensitivity of a function f f: : F is defined
A max adjacent( DD)) [I f(D')l
(2)
where adjacent( D. D*) implies D' E D are neighboring datasets.
Definition (Laplace mechanism): The Laplace mechanism M (Sentral)
DD D IR is randomized
algorithm defined as
M (D,f(), = + ( sk)
(3)
for D, Laplace (0, Af/c). and some transformation function f R
The Laplace mechanism induces e- c-CDP; see Dwork & Roth (2014) for proof.
While CDP relies on trusted database administrator, LDP provides much stricter guarantee in which the individual does not need to trust an administrator. Instead individuals are able to privatize their data before sending it using local randomized algorithm.
Definition ((e, 0)-local differential privacy): A local randomized algorithm A: X' + 2, that takes as input a datapoint from the data domain X' satisfies (e, ol-ccaal differential privacy if for 6, 2 0, for all S S 2, and for any inputs x' E X.
p(A(r) E S) & (c) E 5) +
(4)
When 8 we say A satisfies e-LDP.
Definition (Local Laplace mechanism): The local Laplace mechanism M X - IR is
randomized algorithm defined
Molall (x, f(), f(x) (s1.....,
(5)
for x E X, Laplace(0, Af/e), and some transformation function f X + 2, where 2 C R and the l, sensitivity f(-) is defined as A = max [lf(x) f(x')l11
The local Laplace mechanism satisfies e-LDP (see Appendix A for proof).
Another common choice of mechanism for privatizing continuous data the Gaussian mechanism, which satisfies (e, >> 0)-LDP, For the remainder of the paper however, we exclusively study the local Laplace mechanism since it provides a strong privacy guarantce (i.c. = 0). We note that our approach could be used to learn Gaussian mechanism with minimal modification.
3 PROPOSED METHOD
Early work on data collection algorithms, such as randomized response, have relied on very simple randomized algorithms to induce privacy. Throughout this paper we have benchmarked our results against such mechanism, in which add Laplace noise all continuous features. and flip each of the categorical features with some probability. By the composition theorem (Dwork & Roth 2014), each feature then contributes towards the overall L.P guarantee of the d-dimensional datapoint as
c= E Q. Since we have no prior knowledge about which features are most important, we choose the Laplace noise level (or flip probability) for each feature to be such that c/d. See Appendix E.2 for further details. As with our approach, this benchmark mechanism can act on any date type, and forms downstream task agnostic LDP version of the data.
As increases, e; decreases for each feature i. The noise required to induce -PD thus grows with data dimensionality. For high-dimensional datasets like images large tables, features are often highly correlated; consequently, noising features independently is wasteful towards privatizing the information content in each datapoint. more effective approach privatization involves noising learned
representation of each datapoint using a generic noising mechanism.
To this end, data. This learned mapping from data space to latent space forms our function f()) in Equation 5 and requires only small unlabelled dataset from similar distribution, which most organizations
VAE-based approach to learn now-dimensional latent representation of our
3

---

Under review as a conference paper at ICLR 2021
will typically already have access to, either internally or publicly. Applying the Laplace mechanism (as described in Section 3.1 thus ensures the encoded latents, as well as reconstructed datapoints, satisfy LDP. We can therefore privatize data at the latent level or the original-featun level; preference between these two options is application specific, and we investigate both experimentally.
Data owners can apply this learned LDP mechanism to their data before sharing, and the database administrator forms an LDP data set from the collected data. This set, along with information on the type of noise added, is used to train downstream machine learning algorithms. Though our privatization method is task agnostic, in this paper we focus on classification tasks in which we have some features z for which we want to predict the corresponding label y. At inference time, we show that this classifier can act on either clean or privatized datapoints, depending on the application.
3.1 LEARNING A LAPLACE MECHANISM
We assume that our data x: is generated by a random process involving latent variable 2. We then optimize a lower bound on the log likelihood (Kingma Welling 2014)
logp(x) = =log / p(= p(=)po 2 (xte)) po(l))) - DRL. (2o()
(6)
where p(z) is the prior distribution and (2(2)) is the approximate posterior. The generative dis- tribution (=]22) and approximate inference distribution qs (=(z)) are parameterized by neural net- works, with learnable parameters 0 and & respectively. While the distributions over latent space are commonly modeled Gaussian, we aim to learn Laplace mechanism and so instead we choose
p(2)
II((t)) and g0(2|>) II (2(lz)
i=1
where p(z;) ~ Laplace(O, 1/v2) and (z/lo)) ~ (x); b).
We parameterize (+(()) with neural network and restrict its output via carefully chosen activation function v()) acting on the final layer (a(-) = (+)).-)). This clips the output (+) such that all points are within a constant l no-norm of the origin, by re-scaling the position vector of points at larger & distance. In this way we ensure that Anp = 2/.
With Aus finite, we note that if we fix the scale sample from our encoder distribution qo (z|z) is equivalent to passing a point æ through the Local Laplace mechanism (llocal) (.)(-),, from Equation 5 Therefore to obtain a representation of æ that satisfies eDpL. we simply have to pass through the encoder mean function (+)()) and add Laplace (0, 21/ez) noise. We refer to this model as variational Laplace mechanism (VLM).
21/ez at inference time, then drawing
We further prove in Appendix B that reconstruction obtained by passing ă through the decoder network also satisfies c, LDD, allowing us to privatize datapoints at either latent level ž, or original- feature level ă.
Note that b is always fixed at inference (i.e. data privatization) time to guarantee ă is LDD. However, we experiment with learning b during training, as well as fixing it to different values.
Certain applications of our model require us to share either the encoder or decoder of the VLM at inference time. If the VLM training data itself contains sensitive information, then the part of the network that gets shared must satisfy CDP with respect to this training data. We found the following two-stage VLM training approach to be helpful these cases:
Stage 1: Train a VLM with encoding distribution (=(2(=) and decoding distribution
Po 2=())2) using non-DP optimizer, such Adam (Kingma & Ba 2015).
Stage 2: training a DP enceddre model, fix 6, and re-train the encoder with new distribu- tion Loyyesuee (=|ar). If training a DP-decoder, fix & and replace the decoder with pomase (x) 2). Optimize Oprivate or Oprivate as appropriate using DP-Adam (Gylberth et al. 2017).
Section 4 and Appendix outline applications in which private VL.M components required.
4

---

Under review as a conference paper at ICLR 2021
3.2 TRAINING ON PRIVATE DATA
For supervised learning we must also privatize our target y. For classification, y/ E [l,..., K) is discrete scalar. To obtain a private label y. flip y with
fixed probability p (K 1)/K to one of the other K - categories: pl = ily = j) = (1 pp)l(i j) p/ (Ktppp/KKKIKK) 1) # j). Setting
p = (K 1) K 1) induces Ey LDDLPP (see Appendix for proof).
C
By the composition theorem (Dwork & Roth 2014), the tuple (ă, ÿ) satisfies e-LDP where = ex Ey. Downstream models may be more robust label noise than feature noise, or vice versa, so for fixed e we set 6x xe and cy (1 A)e, where À is chosen to maximise the utility of the dataset. practice, we treat
model hyperparameter.
Rather than training the classifier (-( directly on private labels, we incorporate the known noise mechanism into our objective function
log p() log Σ (x y=1
log log Σ P
y=1
depending on whether we choose work on feature level. or latent level.
At inference time we can classify privatized points using Pe (y|z) or (=ll).). We also show empir- ically that we can classify clean points using the same classifier. We refer these tasks as private and clean classification, with applications given in Sections 4 and
3.3 PRIVATE VALIDATION AND HYPER-PARAMETTE OPTIMIZATION
Typically for model validation, one needs access to clean labels y (and clean data æ for validating a clean classifier). However, note that we need only collect privatized model performance metrics on test and validation sets, rather than actually collect the raw datapoints.
To do this, send the trained classifier to members of the validation set that they can test whether it classified correctly: E 10. They return an answer, flipped with probability = 1/(e* + 1)
such that the flipped answer satisfies e-LDP, and we estimate true validation set accuracy A
Noi EN on from privatized accuracy À
N.a ENai čn using
A A-p
- 2p
Warner 1965). We use this method to implement a grid search over hyperparameters of our model.
4 CLASSIFYING CLEAN DATAPOINTS: APPLICATIONS AND EXPERIMENTS
Below we demonstrate the versatility of our model by outlining a non-exhaustive list of potential applications, with corresponding experiments. Experiments trained on the MNIST dataset (Le- Cun et al. 1998), or the Lending Club [ The CDP requirements differ between applications, but are explicitly stated for each application Appendix D For all stated (e, 6)-CDP results 5 = 10-5, whilst for 8)-DP results, o = 0. All results quoted are the mean of trials; error bars represent +1 standard deviation. Appendix E describes experimental setup and dataset information.
In Sections 4.1 and 4.2 we investigate the clean classification task, and report on clean accuracy. Namely, the classifiers are trained on privatized data in order classify clean datapoints inference time. We also study the classification of privatized datapoints, using classifier trained on privatized data, which we refer to as private classification (and report private accuracy) in Section
4.1 DATA COLLECTION
Organizations may have access to some (potentially unlabelled) clean, internal data D1, but want to collect privatized labelled data D:, in order to train a machine learning algorithm. For example, a public health body may have access to public medical images, but want train diagnosis classifier
https: www. kaggle. wwttwwwwwwwwettttt for se/1end ccuubb

---

Under review as a conference paper at ICLR 2021
to be used in hospitals using labelled data collected privately from their patients. Similarly, tech company with access to data from a small group of users may want to train an in-app classifier; to do so they could collect private labelled training data from broader group of users, before pushing the trained classifier to the app. Finally, a multinational company may be allowed to collect raw data on their US users, but only LDP data users from countries with more restrictive data privacy laws.
In this experiment, we split the data such that Dj and D2 follow the same data distribution, however in practice this may not always be the case. For example, when D2 is sales data collected in different time period, or user data collected in a different region, we may expect the distribution to change. We have omitted such an experiment here, but the extreme case of this distributional shift is explored experimentally in Section 4.2
We run this experiment on both MNIST and Lending Club. As in Sections 3.1 3.3 we first train a VLM with DP encoder using D1, then privatize all datapoints and corresponding labels in D2 before training a classifier on this privatized training set. Results are shown in Figure [
For both MNIST and Lending Club, we significantly outperform the baseline approach of noising pixels directly. The benchmark performed at random accuracy for all local S 100 for MNIST (local s 20 for Lending Club). Our model performed well above random for all local values tested.
For MNIST, we see that latent-level classification outperforms feature-level classification for higher local-e values. Indeed, the data processing inequality states we cannot gain more information about given datapoint by passing it through our decoder. However at lower local we see feature level classification accuracy is higher. We hypothesize that at this point, so much noise is added to the latent that the latent level classifier struggles, while on pixel level the VLM decoder improves classification by adding global dataset information from Dj to the privatized point.
For Lending Club, we do not see clear difference between latent-level and feature-level accuracy- However we also note that the features are not as highly correlated as in MNIST, so perhaps the decoder has less influence on results.
Finally, we see that reducing central e has an adverse effect on MNIST classification accuracy. especially for higher local e. The effect seems negligible for Lending Club and we hypothesize that this is due to the large quantity of training data, along with the easier task of binary classification.
4.2 NOVEL-CLASS CLASSIFICATION
As discussed in Section[4.1] the internal data Dj and the data to be collected D2, may follow different data distributions. In the extreme case, the desired task on D2 may be to predict membership in a class that is not even present in dataset Dj. For example, in a medical application there may be large existing dataset Dj of chest scans, but only relatively small dataset D2 that contains patients with a novel disease. As before, public health body may want to train a novel-disease classifier to distribute to hospitals. Similarly, a software developer may have access to an existing dataset D,, but want to predict software usage data for D2, whose label is specific the UI of a new release.
We run this experiment on MNIST, where the internal D2 contains training images from classes 0 to 8, (with a small number of images held out for classification), and D: contains all training images from class 9. As in Sections 3.1 and 3.2 we first train a VI.M with a DP encoder on Di, then privatize all images D2 (we are not required collect or privatize the label since all images have the same label). We then train a binary classifier on the dataset formed of the private 9's and the held out internal images from classes 0-8 (which we privatize and label 'not 9's').
On both latent and feature level, we obtain approximately 75% accuracy at local e = 2 and above random performance at local c = 1. The benchmark of noising features directly achieves random performance for all local 100 (not shown in the figure). Once again, the effects of reducing central e appear greater at higher local values, and we see that the latent-level classifier outperforms the feature-level classifier high local e, but not at lower local e.
Results are shown in Figure 2

---

Under review as a conference paper at ICLR 2021
MNIST Latent-Level Classitication
MNIST Feature-Level Classification
Clean accurary
Clesn accuracy
1.0
Our method,
1.0
Our method, Ecroul
mothod, Koennsl
method, Eceemal
Our method,
Our method, Evenal
0.8
Noise festures directly
0.8
Noise festures directly
Random classifier
Random classifier
0.2
02
10
10
6
Local
Local
Lending Latent-Level Classification
Lending Festure-Level Classification
Clean accuracy
Clean accuracy
Our method, Rcuntra
Our method, Eeesta
Our method,
Our method, m
0.70
Our method, freera =
0.70
Our method, Frema" =
Noise directly
Notse features directly
0.65
Random classiffer
0.65
Random classiffer
0.60
0.60
0.50
0.50
10
10
Local
Local
Figure 1: Clean accuracy as a function of local e for data collection. Results are shown for the MNIST dataset (top) and Lending Club (bottom), on latent level (left) and feature level (right). Each line indicates a different value of d = 10-9)-CDP at which the encoder was trained. The r-axis shows the e-LDP guarantee for the collected training set.
Clean accuraty
Our method. Camoo
Clean accuracy
Our method, Fantral
method,
method,
Our method, Fowerol
Our methad, fontt 1 directly
0.9
directly
0.9
Random classifier
Random classifier
0.7
0.7
0.5
0.5
Figure 2: Clean accuracy as function local for novel-class classification on latent level (left) and feature level (right). Each line indicates different value of (c, 10-5)-CDP at which the encoder was trained. The r-axis shows e-LDP guarantee for the collected training set.
4.3
DATA JOINING
An organization training a classifier on some labelled dataset D could potentially improve perfor- mance by augmenting their dataset with other informative features, and so may want to join D with features from another dataset D2. We assume the owner of D2 may only willing to share a privatized version of this dataset. For example, two organizations with mutual interests, such the IRS and a private bank, or fitness tracking company and a hospital, may want to join datasets to improve the performance of their algorithm. Similarly, it may be illegal for multinational organiza-

---

Under review as a conference paper at ICLR 2021
Somi prvvte accuracy
Pluvte accuracy
Our method
Our method, Scutrul =
Noise features directly
1.0
Our method, Scotru =
Only clean teatures
Our method, foreral #1
0.64
0.8
Noise features directly
Theoretical upper bound
0.62
Random clessifier
0.6
0.60
0.4
0.58
0.2
0.56
10
Local
Local
Figure 3: "Semiiprivate"" accuracy versus local Figure 4: Private accuracy versus local с for for data joining (private features shared and latent-level data collection. Each line indicates joined on latent level). The z-axis shows the e- LDP guarantee for the collected training set.
a different value of (e, = 10-$)-CDP at which the encoder was trained. The x-axis shows the e-LDP guarantee for the collected training set.
tions to share and join non- non-privatized client data between departments in different regions, but legal do so when the shared data satisfies LDP.
We run this experiment on Lending Club, where we divide the dataset slightly differently from previous experiments: both datasets contain all rows, but Dj contains a subset of (clean) features, along with the clean label, and D2 contains the remaining features (to be privatized).
We follow a privatization procedure similar to that of Section [31] with the distinction that the VLM should be both trained on D2, and used to privatize D2. For the classification problem, instead of Equation g optimize log Pv (x2) where (x11, 31) Dj and .P2 E D(prrivtt)) We not required to conduct private grid search over hyperparmeters as Section 3.3 since we have access to all raw data needed for validation. Note that unlike the previous two experiments, train the classifier on a combination of both clean and privatized features, and we classify this same 'semi-privatee group of features at inference time.
Results are shown in Figure 3 We can see that using features from D2 only, we obtain classifica- tion accuracy of 56.1% 1%, while classifying on all (clean) features, we obtain 65.8% accuracy. The benchmark of noising the D2 features directly never achieves more than percentage point accuracy increase over classifying the clean features only, whereas our model achieves a significant improve- ment for local E [4, 10). We share the privatized features on latent level in this experiment and so do not need to satisfy CDP.
CLASSIFYING PRIVATE DATAPOINTS: APPLICATIONS AND EXPERIMENTS
In Sections 4.1 and 4.2 we have been investigating the use of privatized training data to train algo- rithms that classify clean datapoints. In some use cases however, we may want train algorithms that act directly on LDP datapoints at inference time. Most notably, in the data collection framework, the organization may want to do inference on individuals whose data they have privately collected.
However from the definition of LDP in Equation 4 it is clear that a considerable amount of in- formation about a datapoint x is lost after privatization, and in fact, elassification performance fundamentally limited. In Appendix (F, we show that for given local c, the accuracy A of K K-class latent-level classifier acting on a privatized datapoint (where the privatization mechanism has latent dimension d 2 K/2) is upper bounded by
K/2-1
Σ
11 ((1/2-)
--
1-j
(-1
e
1+j
j//2
e-1/21)
2
E+1
(K - 2) e/12
(10)
J=0
In Figure 4 we show the accuracy of our model from Section 4.1 (data collection) when applied to privatized datapoints at inference time, and compare to the upper bound in Equation T0]
8

---

Under review as a conference paper at ICLR 2021
Running this experiment MNIST, privatized datapoints, compared with clean classification results.
considerable drop in performance when classifying
We are clearly not saturating the bound from Equation 10. While it may initially appear that priva-
model is performir.....gg.... we note that our model aims to build a hownstream-lask-agnae tized representation of the data. This means that the representation must contain more information than just the class label. Meanwhile, the upper bound is derived from the extreme setting in which the latent encodes only class information, and would be unable solve any other downstream task.
Though Equation 10 is constructed under the framework of latent-level classification, we do note that feature-level classifier seems marginally outperform the latent-level one (see Appendix G This may be a result of the decoder de-noising the latent to some extent.
CONCLUSION AND FUTURE WORK
In this paper have introduced framework for collecting and sharing data under the strict guar- antees of LDP. We induce LDP by learning to efficiently add noise to any data type for which representation learning possible. We have demonstrated a number of different applications of our framework, spanning important issues such medical diagnosis, financial crime detection, and customer experience improvement, significantly outperforming existing baselines on each of these.
This is the first use of latent-variable models for learning LDP Laplace mechanisms. We foresee that even stronger performance could be achieved by combining our method with state-of-the-an -frt latent-variable models that utilise more complex architectures, and often deep hierarchies of latents (Gulrajani et al. 2017: Maaløe et al. 2019 Ho et al. 2019). this work we sought to show that significant hurdles in LDP for high- nign-dime data could be overcome using representation- learning driven randomization algorithm, which we indeed think is well established in this paper.
REFERENCES
M. Abadi, A. Chu, I. Goodfellow, H. B. McMahan, I. Mironov, K. Talwar, and L. Zhang. Deep learn- ing with differential privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, 2016.
G. Acs, L. Melis, C. Castelluccia, and E. De Cristofaro. Differentially private mixture of generative neural networks. IEEE Transactions on Knowledge and Data Engineering, 31(6):100--122) 2019.
N. Agarwal, A. T. Suresh, F. Yu, S. Kumar, and B. McMahan. cpSGD: Communication--eficienn and differentiallyypriieeee distributed SGD. In Advances in Neural Information Processing Systems 31, 2018.
B. Ding, J. Kulkarni, and Yekhanin. Collecting telemetry data privately. Advances in Neural Information Processing Systems 30. 2017.
J. C. Duchi, M. I. Jordan, and M. J. Wainwright. Local privacy and statistical minimax rates. In IEEE 54th Annual Symposium on Foundations of Computer Science, 2013.
C. Dwork and A. Roth. The algorithmic foundations of differential privacy. 2014.
C. Dwork, F. McSherry, K. Nissim, and A. Smith. Calibrating noise to sensitivity in private data analysis. In Theory of Cryptography, 2006.
Ó. Erlingsson, V. Pihur, and A. Korolova. RAPPOR: Randomized aggregatable
ordinal response. In Proceedings of the 2014 ACM SIGSAC Conference on Computer and Com- munications Security, 2014.
I. Gulrajani, K. Kumar, F. Ahmed, A. A. Taiga, F. Visin, D. Vazquez, and A. Courville. PixeIVAE: A Latent Variable Model for Natural Images. In International Conference on Learning Represen- fations, 2017.

---

Under review as a conference paper at ICLR 2021
R. Gylberth, R. Adnan, S. Yazid, and T. Basaruddin. Differentially private optimization algorithms for deep neural networks. In International Conference on Advanced Computer Science and Infor- mation Systems (ICACSIS), 2017.
J. Ho, X. Chen, A. Srinivas, Y. Duan, and P. Abbeel. Flow++: Improving flow-based generative models with variational dequantization and architecture design. In International Conference on Machine Learning, 2019.
Jolly. FCA admits revealing confidential details of 1,600 consumers. In The Guardian, 2020.
P. Kasiviswanathan, H. K. Lee, K. Nissim, S. Raskhodnikova, and A. Smith. What can we learn privately? In 49th Annual IEEE Symposium on Foundations of Computer Science, 2008.
D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations, 2015.
D. P. Kingma and M. Welling. Auto-Encoding Variational Bayes. In International Conference on Learning Representations, 2014.
Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 1998.
L. Maalee, M. Fraccaro, V. Liévin, and O. Winther. Biva: A very deep hierarchy of latent variables for generative modeling. In Advances in Neural Information Processing Systems 32. 2019.
B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. Aguera y Arcas. Communication-Efiien Learning of Deep Networks from Decentralized Data. In Conference on Artificial Intelligence and Statistics, 2017.
N. Papernot,
N. Abadi, U. Erlingsson, I. Goodfellow, and K. Talwar. Semi- cumi-supervised knowledge transfer for deep learning from private training data. In International Conference on Learning Representations, 2017.
X. Ren, C. Yu, W. Yu, S. Yang, Yang. A. McCann, and S. Yu. LoPub: High-dimensional crowdsourced data publication with local differential privacy, In IEEE Transactions Informa- tion Forensics and Security, 2018.
D. Rezende, S. Mohamed, and D. Wierstra. Stochastic Backpropagation and Approximate Infer- in Deep Generative Models. In International Conference on Machine Learning, 2014.
N. Rodríguez-Barroso, G. Stipeich, D. Jiménez-Lópezz J. A. Ruiz-Millán, E. Martínez-Cámn G. González-Seco, M. V. Luzón, M. Ángel Veganzones, and F. Herrera. Federated Learning and Differential Privacy: Software tools analysis, the Sherpa.ai FL framework and methodological guidelines for preserving data privacy. arXiv preprint, arXiv:2007.00914 2020.
I. Sweeney. Weaving technology and policy together maintain confidentiality. In The Journal of Law, Medicine & Ethics, 1997.
S. Takagi, T. Takahashi, Y. Cao, and M. Yoshikawa. P3GM: Private high-dimensional data release via privacy preserving phased generative model. In JEEE International Conference on Data En- gineering. 2021.
J. Tang, A. Korolova, X. Bai, X. Wang, and X. Wang. Privacy loss in Apple's implementation of differential privacy on MacOS 10.12. arXiv preprint. arXiv:1709.0275. 2017.
A. Triastcyn and B. Faltings. Generating artificial data for private deep learning. In Proceedings of the PAL: Privacy-nnhnccnng Artificial Intelligence and Language Technologies, AAAI Spring Symposium Series, 2019.
L. Warner. Randomized response: A survey technique for eliminating evasive answer bias. Jour- nal of the American Statistical Association, 1965.
L. Xie, K. Lin, S. Wang, F. Wang, and J. Zhou. Differentially private generative adversarial network. arXiv preprint, arXiv: 06739 2018.
10

---

Under review as a conference paper at ICLR 2021
A
PROOF THAT THE LOCAL LAPLACE MECHANISM SATISFIES LDP
Claim: The Local Laplace mechanism satisfies e-local differential privacy.
Proof:
We follow an approach similar to the proof in Dwork & Roth (2014) that the Laplace Mechanism satisfies CDP. Assume and x' are two arbitrary datapoints. Denote M (z) = fz) +
where ~ Laplaco(0,, Af/c). Then for some arbitrary c we
know that
p(Mblcal (x) MMMocaa (x) ci) (Mllca(sz) c)
(11)
p(M local (I') = c2)
exp
Df
(12)
=1/1(0))aa
i=1
exp
Af
k
((([i(v')) cil (f((x) cal)
exp
(13)
Af
i=1
[e]s((a)= - M((r
exp
(14)
Af
1=1
[clf(x') f(x)111
exp
(15)
A
S exp (E)
(16)
where the first inequality comes from the triangle inequality, and the second comes from the defini- tion of f
B PROOF THAT DECODED PRIVATE LATENTS SATISFY LDP
Claim: If point in latent space satisfies e-LDP, then this point still satisfies e-LDP after being passed through a deterministic function, such as the function that parameterizes the mean of the decoder network.
Proof: We follow approach similar to the proof that central differential privacy is immune to post-processing (Dwork Roth 2014). Consider an arbitrary deterministic mapping g : + Let S C X and T = {z E Z g(2) E S}. Then
p(g(A(z)) S) = p(A(z) ET)
(17)
S < (c) (()((') TT)
(18)
= (xxp())(((A(e))) E
(19)
C PROOF THAT THE FLIP MECHANISM SATISFIES LDP
Claim: The flip mechanism pūū dy 1) p)[(i 5) p/(K 1))[(i # () where
p= = (K 1)/(e' + 1) satisfies c-local differential privacy.
11

---

Under review as a conference paper at ICLR 2021
Proof: We can write, for any j.j.
p ily = 5)
(1 - -)
(20)
p = fly = jr)
(1 (1-p) = (K 1)1(+
(K-1)1-P)
(21)
(K-1)(1-))
1
otherwise
e
i=jiサ}
e
i jン
(22)
1
otherwise
Therefore, we have that for any i,j,J', p(ú=ily=i)
Se.
-(ū=ilys)")
D IMPLEMENTATION REQUIREMENTS FOR DIFFERENT APPLICATIONS
In the scenario that Dj contains sensitive information, the encoder or decoder may need to be trained with the two-stage approach (outlined in Section in order guarantee (e. 3 wrt Dj. There are broadly two scenarios in which this is the case. Firstly, if private data is published on pixel level then a DP decoder required. Secondly, the encoder needs to be shared with the client (for example, in client side data collection) then DP encoder is required. For pixel level data collection a DP decoder is not required, since the client can share their privatized latents and these can be 'decoded' to pixel level on the server side, avoiding the need to share the decoder. Taxpec[exl outlines the CDP requirements for the applications discussed in our paper.
Table 1:
Central differential privacy requirements for the VLM, with respect to the dataset D1-
Application
Data Shared
Central-DP Requirement
Data Collection
Feature or latent level
DP-Encoder
Data Joining
Feature level
DP-Decoder
Latent level
None
Novel-Class Classification
Feature or latent level
DP-Encoder
E EXPERIMENTAL SET UP
For every experiment in the paper, we conducted three trials, and calculate the mean and standard deviation of accuracy for each set of trials. The error bars represent one standard deviation above and below the mean.
We use the MNIST dataset and the Lending Club dataset. MNIST a dataset containing 70,000 images of handwritten digits from 0-9 with corresponding class labels; the task is 10-way classifica- tion to determine the digit number. Lending Club is a tabular, financial dataset made up of around 540,000 entries with 23 continuous and categorical features (after pre- pre-processing before one-hot encoding); the task is binary classification, to determine whether a debt will be re-paid.
E.1 DATA PRO-PESCEGIING
For MNIST, we converted the images into values between 0 and by dividing each pixel value by 255. These are then passed through logit and treated as continuous.
For Lending Club, number of standard pre-processing steps are performed, including:
Dropping features that contain too many missing values or would not normally be available to a loan company.
12

---

Under review as a conference paper at ICLR 2021
Mean imputation to fill remaining missing values.
Standard scaling of continuous features. Extreme outliers (those with features more than 10 standard deviations from the mean) are removed here.
Balancing the target classes by dropping the excess class entries.
One-hot encoding categorical variables.
The target variable denotes whether the loan has been charged off not, resulting in binary classification task. The train, validation, test split is done chronologically according to the feature 'issue date'.
In real world applications the sizes of the VLM training and validation sets, and the classifier training and validation sets would be pre-determined. For our experiments we used the data splits outlined in the following Sections.
E.1.1 DATA COLLECTION
MNIST: The MNIST dataset contains 60,000 training points and 10,000 test points. We split both sets using 75% for the VI.M and the remainder for the classifier. The VL.M test set used for validation since no test set is required here. The classifier training points are split randomly in 9:1 ratio to form training and validation sets. We report classifier performance on the classifier test set.
This dataset is split into train, validation and test sets according to the issue date of loans. The oldest 85% of data forms training data, with the remaining forming the validation and test data. As with MNIST, training data.
Lending Club:
75% for the split between VLM training data and classifier
E.1.2 NOVEL-CLAS CLASSIFICATION
We use a similar approach to the above, but split the data between the VLM and the classifier such that the VLM train/validation sets contain ths of (unlabelled) training images from classes 0 to 8. The remaining ths of to 8 images, and 9s, are used for classifier train, test and validation sets. Our VLM datasets then contain equal class balance for the classes 0 to 8, and the classifier datasets contain equal class balance for and "not 9s*
MNIST:
E.1.3 DATA JOIN
Lending Club:
For this experiment, the datasets were split between the VLM and the classifier column-wise, between the dataset's 23 features. The VLM datasets contain 8 features (month of earliest credit line, number of open credit lines, initial listing status of loan, application type, address (US state), home ownership status, employment length, public record bankruptcies). The classifier datasets contain the remaining features. The feature split was chosen such that the first dataset contains information solve the classification task, but the features from the second dataset contain information which, at least before privatization, further improve classifier performance.
F.2 NOISING FEATURES DIRECTLY
For continuous features, we assume that A is equal to the difference between the maximum and minimum value of that feature within the training and validation sets used to train the VLM in the main experiments, after pre-processing. One then has to clip any values that lie outside this interval in the shared/collected dataset at privatization time.
E.3 HYPERPARAMETER CHOICES
We conducted grid search number of the hyperparameters in our model, in order find the optimal experimental setup.
For stage 1 of the VLM training, learning rate of 10-4 4 and batch size of 128 used for Lending Club experiments, and learning rate of 5 x 10-4 and batch size of 64 was used for MNIST. We then searched over the following model hyperparameters (with central c=oo):
13

---

Under review as a conference paper at ICLR 2021
Table 2: DP-Adam hyperparameters used for data collection and novel-class classification.
Task
Central
Learning Rate
Batch Size
Noise Multiplier
MNIST
5
5e-4
0.7
1
5e-4
64
1.1
Lending Club 5
le-4
128
0.56
le-4
128
1.1
Table 3: VLM hyperparameters used for data joia experiments.
Experiment
Task
Local
1
pr training
Data Joining
Lending Club
5
20
10
5
10
8
5
15
6
5
15
4
5
10
2
5
5
15
5
10
The proportion of our privacy budget assigned to the datapoint, compared with the label i.e. x == cz/ + (y).
The ε clipping distance our inference network mean i.e. Aps/2.
The Laplace distribution scale b of our approximate posterior distribution during pre- training of the VLM. Note that we report this as the 6pre-traiingg-LD value induced by a sample from this posterior distribution, given in the previous point i.c. Epre trainigg 21/b. This is fixed throughout training, unless 'learned' is specified the below table, in which case the parameter is learned scalar.
The latent dimension d. This was fixed to 8 for data collection experiments but we searched over d E {5,8) }} for the data join experiments due the smaller number of features.
We also did a grid search over the following DP-Adam hyperparameters for central {1,5):
Noise multiplier
Batch size
DP learning rate
The DP-Adam hyperparameter max gradient norm was fixed to throughout. The number of train- ing epochs needed to reach the target central value follows from the choice of hyperparameters, combined with the VLM training set size (45,000 for MNIST, and 341,000 for Lending Club). Note that we fixed 10-5 5 for all experiments.
The results from these grid searches are given in Tables and
E.4 NETWORK ARCHITECTURES
Throughout the paper, we used feedforward architectures for both the VLM and classifier networks.
For MNIST, use VLM encoder network with 3 hidden layers of size {400, 150. 50), and decoder network with 3 hidden layers size {50, 150, 400). For Lending Club, we use VLM encoder and decoder network with 2 hidden layers of size {500, 500). For the the latent level classifier we used a network with hidden layer of size 50 and for pixel level classifier we use network with 3 hidden layers of size (400, 150, 50).
14

---

Under review as a conference paper at ICLR 2021
22
Decision boindhiry
11)
Figure 5: The decision boundary for classifier that equally separates (in (--distance) vertices c(t)
for E 21, 4} in 2-dimensional space.
F PROOF OF UPPER BOUND ON NOISY ACCURACY
In this section, derive upper bound accuracy for the classification of datapoints privatized using the Laplace mechanism (see Equation 4). To simplify the proof, we make the following assumptions:
We have K equally balanced classes.
is even.
< 2d where is the dimension the output of f(-) (the latent space on which we add Laplace noise).
These are true for all experiments in this paper.
First, suppose that K 2d. Since we add iid Laplace noise to each datapoint, we will obtain the highest possible accuracy when ʃ(-) maps datapoints from class as far away from datapoints from class & as possible. This maximum separation distance can be at most Aʃ; we can separate all K classes by distance A in our d dimensional latent space iff each class is mapped to a separate vertex e(x) of the taxicab sphere of (fo-nor) radius Af/2.
The decision boundary is given by the line that equally separates these vertices in fi-space, as shown (for 2 dimensions) in Figure
The accuracy of the classifier C on datapoints privatized by the latent-space Laplace mechanism ((11x) (a)) is given by
A E (eww)-p(eos)
y)
(23)
E Eyrpi (C(cl)) +6) =
(24)
= (C( +s)= =1)
(25)
where s = ........ sa) and s, Laplace(O,, Af/e). The second equality follows from the fact that all points from given class are mapped the same point in latent space, and the final equality follows from the symmetry between classes. This final term gives the probability that when we
add Laplace noise to c(I) and obtain the private representation a(1), we do not cross the decision boundary. We assume WLOG that c(1) = (1,0...., and calculate this probability as follows (dropping the superscript for clarity)
15

---

Under review as a conference paper at ICLR 2021
y
Decision beandory
e(2)
Decision boundary
c(2)
c1)
(a)
(b)
Figure 6: The shaded areas represent, for = 2 and K = 2, the decision boundaries for: (a) a function f() that maps the two classes onto opposing vertices: (b) a function f()) that maps the two classes onto adjacent vertices. Refer to Appendix F for details on the color-coding of shaded areas.
(-
||ē c|li
A
dēg ... dēd (26)d exp
(26)
6130, & (clii
b
[2
1
|
dē
exp
dē. 2h
exp
(27)
2b
b
1
[2] 1
d-1
dc1
exp
1 exp
(-01/b)
(28)
26
b
do 2b
exp
121
d 1
(-1>
exp
(-
i
(29)
b
j
j=0
Σ d 1-11 1) (-13² /0
1
[ -1 exp
dē 2b
exp
(30)
b
Σ (d- (²= (-17)
1/'
(j - 1) +1
dē 2b
exp
b
1
Ti + 1)
+
dē 2b
exp
(31)
b
1-d
d
1+ 2
exp
(-1/b)+ +
Σ
d-1
d
(-1)3
exp (-j/b)
cxp (-1/b)
(32)
26
j
1 1-
2(1 - j)
j=0.j
(1 - d) E +1
exp
(-c/2) +
Σ
d-1
d dd
exp (-je/2) (-1)3
exp (-e/2)
(33)
4
j
1-
2(1 2(1- -)
f=0
where in the last step we used the fact that in this case e = 2/b. By substituting d K/2, Equa- tion 10 follows directly.
Now, we consider the case K < 2d. Clearly, the taxicab sphere has more than K vertices, and so classes can occupy different combinations of vertices. The maximum accuracy will be found where the occupied vertices are maximally separated from each other. For the case of Laplace noise, where probability mass decreases exponentially with e distance from the mean, it is clear
16

---

Under review as a conference paper at ICLR 2021
that the probability of noised datapoint crossing decision boundary is higher when the classes are centered on vertices aligned along different axes. We have shown this for in Figures ((a) and [6(b] where clearly the probability mass of the blue shaded area for Laplace distribution with mean cl-) is larger than the green shaded area. Therefore we are more likely to cross the decision boundary Figure 6(b) given fixed quantity of noise.
Thus, the optimal setup is when the K classes are positioned vertices aligned along the first K/2 axes. Noise added to the remaining d K/2 dimensions does not affect the classifier, and so Equation still holds.
G FEATURE LEVEL PRIVATE CLASSIFIER FOR DATA COLLECTION
Figure 7 shows the private accuracy results for MNIST data collection, on latent level.
Pricate accuracy
1.0
od, fzama
Our method, =5
method, Ervers
0.8
Noise features directly
Theoretical upper bound
Random clessifier
0.6
0.4
0.2
Local
Figure 7: Private accuracy function of local for data collection (feature level). Each line indicates different value of (e, d = 10 10-5)-CDP at which the encoder was trained, each point on the æ-axis shows the e-LDP guarantee for the collected training set.
17

---

Under review as a conference paper at ICLR 2021
Table 4: VLM hyperparameters used for data collection and novel-class classification.
Experiment
Task
Local
A
Fpr training
Clean Accuracy,
MNIST
N/A
10
learned
Latent Level
0.7
27
Classification
0.7
5
29
0.7
5
15
0.7
7.5
2
0.7
7.5
21
0.7
5
15
Lending Club
0
N/A
10
learned
10
0.7
5
15
0.7
5.
29
0.7
5.
29
4
0.7
5
15
0.95
0.95
10
21
Private Accuracy, Latent Level Classification
MNIST
00
N/A
10
learned
10
0.7
10
5
0.7
7.5
0.7
7.5
4
0.7
5
5
0.7
15
0.7
7.5
15
Lending Club
0
N/A
10
learned
0.95
15
0.95
5.
15
0.7
5.
15
0.7
5
15
2
0.95
5
29
0.95
10
21
Clean Accuracy, Feature Level Classification
MNIST
0
N/A
10
learned
10
0.7
5
29
8
0.7
5
15
0.7
7.5
21
0.7
5
0.7
7.5
5
0.7
15
Lending Club
0
N/A
10
learned
10
0.7
5
29
0.95
5.
29
0.7
5.
15
0.7
15
0.7
15
0.7
10
15
Private Accuracy, Feature Level Classification
MNIST
8
N/A
10
learned
10
0.7
10
0.7
10
0.7
10
5
0.7
5
0.7
7.5
0.95
5
15
Lending Club
0
N/A
10
learned
10
0.95
5
15
0.95
5.
15
0.95
5.
0.7
5
15
0.95
7.5
27
0.95
15
18


Reviews:
[{"review_id": "wYQfuiwrdPP", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In this paper, the authors present a generative-model-based Laplace mechanism. By training the VAE on some dataset, the trained encoder can be used to privatize raw data towards epsilon, delta-LDP. Though the method is novel, the privacy guarantee of the proposed method is not clearly stated and proved. Related experiments are not convincing, either.\n\n**Strength**\nThe paper is well written with a clear motivation, explanation of methodology. To my knowledge, I believe the work is useful for the privacy research community. The proposed method is also novel.\n\n**Weakness**\n- The motivation to use the Laplace mechanism is not very clear. At the beginning of Sec. 2, the authors reason the usage by \"as it provides strong theoretical privacy guarantees\". This is not convincing for readers especially for those who are not familiar with LDP. Since the Laplace mechanism directly comes from the CDP, I would wonder how does the Gaussian mechanism works. How does the Laplace mechanism guarantee privacy better than the Gaussian mechanism? Reference or proof is essential here.\n\n- In page 3, the authors briefly mention that the local version of the Laplace mechanism can be epsilon-LDP if the sensitivity is accordingly defined. This really lacks rigorousness. In the following sections, the authors refer to (Dwork and Roth, 2014) for the post-processing theorem. Since the work (Dwork and Roth, 2014) is mainly about CDP, I am not sure how the post-processing theorem can be adopted for LDP. Either reference or clear proof is required.\n\n- Meanwhile, there lacks an end-to-end proof of the privacy guarantee of the VAE. I am not sure if the proposed VLM training guarantees privacy. Either, the privacy of encoding is not very clear. Especially, there involves a non-private training on stage 1.\n\n- The experiments are run with pretty week baselines. Through this paper, the authors actively use the same conclusion from CDP (Dwork and Roth, 2014). Thus, I suppose the state-of-the-art CDP algorithms should also be applicable to the experimented tasks, e.g, classification. For the specific task, how well is the proposed compared to the SOTA CDP private learning algorithms? For example, (Abadi, et al., 2016), or (Phan, et al. 2017). Especially, (Phan, et al. 2017) also proposed an adaptive Laplace mechanism without depending on pre-training of the mechanism.\n\n- In page 4, the DP-Adam mentioned in Stage 2 is not stated or proved in (Abadi et al., 2016). Only DP-SGD was discussed. A strict proof is required for the DP-Adam which intensively re-uses private results to help improve the gradients. Thus, the privacy guarantee is not straightforward.\n\n- Seems the VLM training is using a non-DP optimizer at stage 1. Then how the whole training could guarantee privacy on the VLM training set. In experiments, the VLM training set is directly extracted from the private dataset (MNIST). Even though the author experiments with diverse D_1 D_2 distribution for VLM train/test in Sec 4.2, the two datasets are still from the same dataset. In practice, when such a D_2 is private, it is hard to find a D_1 to be non-private. I am afraid this could cause serious privacy leakage. Therefore, I doubt if the experimental results are useful for proving the effectiveness of a private algorithm. More realistic dates should be used.\n\n- In Sec 4.1, the authors run the experiments in two steps. First, the VLM is trained with 'a DP encoder using D_1'. I am not clear how the DP encoder comes from. Does the VLM is also trained with DP? The setting has to be clarified.\n\n- The experiment comparison seems not fair for baselines. For VLM, there are two datasets for training VLM and encoding classification train data. However, the baseline only has classification training data. The VLM encoder has additional information about the data distribution or the noise (by back-propagation in VLM training). The unfairness in the information could be the core reason for the difference in performance. How does the baseline perform if it is pre-trained and tuned (hyper-parameters) on another dataset?\n\n(Phan, et al. 2017). Adaptive Laplace Mechanism: Differential Privacy Preservation in Deep Learning", "extra_scores": {}, "extra_text": {}}, {"review_id": "f_mbRtHTzik", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "For LDP, when applying noise directly to high-dimensional data, the required noise entirely destroys data utility. In this paper, authors introduce a novel, application-agnostic privatization mechanism that leverages representation learning to overcome the prohibitive noise requirements of direct methods. They further demonstrate that this privatization mechanism can be used to train machine learning algorithms across a range of applications. They achieve significant gains in performance for high-dimensional data. \n\nIn this paper, authors have benchmarked results against such a mechanism, in which add Laplace noise to all continuous features, and flip each of the categorical features with some probability. \nFor high-dimensional datasets, features are often highly correlated; consequently, noising features independently is wasteful towards privatizing the information content in each datapoint. A more effective approach to privatization involves noising a learned lower-dimensional representation of each datapoint using a generic noising mechanism. Applying the Laplace mechanism thus ensures the encoded latents, as well as reconstructed datapoints satisfy LDP. They focus on classification tasks. At inference time, they show that this classififier can act on either clean or privatized datapoints,.\n\nFor the writing, it’s better to give a clear algorithm. For the experiment, when epsilon<=10, the accuracy is not very good. The related work and comparisons are not enough. They are quite a few work about LDP learning can be literature reviewed. By the way, we usually use private data rather than privatized data.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "MVl-mSVeuBm", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Strong point 1: The idea of putting noise insertion (via noisy data-generation models) and optimization of good representations together to obtain LDP representations and/or synthetic data seems to be effective. While (6) relies on some independency assumptions, it might be fine in most cases and empirical evidence is reported to support it\n\nStrong point 2: It is an application-agnostic approach and theoretically any downstream tasks and models can be supported... When there is a label, the privacy budget is split and random perturbation is used on labels\n\nStrong point 3: It outperforms naive LDP baselines (with noise added directly to features) a lot in experiments\n\nWeak point 1: The proof of the most important result is missing: It is said that \"sampling from $q_\\phi(z|x)$ produces a representation $\\tilde z$ of $x$ that satisfies $\\epsilon$-LDP. I don't think it is a trivial result and the author needs to everything together (including the analysis of sensitivity, the optimization algorithm, and so on) to formally prove it\n\nWeak point 2: A minor issue: in figures of experiments, by \"clean accuracy\", do you actually mean \"accuracy\" (for some algorithms in the figures, it is privacy accuracy?)\n\nW1 is the main reason for the rating of 6 but not higher ones - highly encourage the authors to fix it before the publication ", "extra_scores": {}, "extra_text": {}}, {"review_id": "8Z9gRfIuMF_", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\nThis paper presents a new privatization mechanism for Local Differential Privacy based on representation learning. The proposed VAE-based method is used for the low-dimensional latent representation of the data and uses the Laplace mechanism to satisfy Local DP. The paper shows this mechanism can be used across various applications such as private data collection, private novel-class classification, data joining, etc.\n\nThe paper is clear and easy to follow. The proposed method provides a great solution for data sharing with the local DP and can be used in many real-world applications. However, some clarification is needed. The experimental results can be improved by adding more baselines. Some important/recent references are also missing.\n\nMajor Comments:\n-         It would be better if the authors add a related work section or extend the literature review paragraph in Introduction and include more recent work in this area and point out how the proposed work advances the state-of-the-art. There are several works on DP for high-dimensional data such as:\nAutoGAN-based Dimension Reduction for Privacy Preservation by Nguyena et al.\nP3GM: Private High-Dimensional Data Release via Privacy Preserving Phased Generative Model by Takagi et al.\nThere are also several existing works on LDP based on VAE. The authors are expected to state the differences between the existing work and the proposed work.\n-         The authors mentioned the DP Synthetic data models need large data, this is also the case for training the VAE in this work. Also, they mentioned these techniques need labeled data. DP-GAN models need access to real data for training (no label is required) and then it can be used indefinitely for generating synthetic data. Please clarify this.\n-         In an existing work (P3GM paper mentioned above), it is shown that VAE’s objective function is too sensitive to the noise of DP-SGD, how the authors tackle this problem? And how it affects the final results?\n-         The baseline methods are limited to \"direct noise features\" only. It would be better if the authors use other techniques such as some recent work on LDP on high dimensional data or other general DP classifiers, DP-SGD, PATE, etc. as the baselines for this experiment.\n-         It would be better if the authors also showed the performance of the proposed method on higher dimensional data (e.g., Lending club has only 23 features)\n\n\nMinor comments:\n-         In the introduction, please list the contribution of this work.\n-         Please define all variables in Eq 5-6.\n-         Please add more up-to-date references.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper considers the problem of private data sharing under local differential privacy. 

(1) it assumes having access to a public unlabeled dataset for learning a VAE, so it reduces the dimensionality in a more meaningful way than simply running PCA. (2) the LDP guarantee is coming from the standard Laplace mechanism and Randomized Responses. (3) then the authors propose how to learn a model based on the privately released (encoded) data which exploits the knowledge of the noise distribution.

None of these components are new as far as I know, nor were they new in the context of differential privacy. For example, the use of a publicly available data for DP was considered in: 

- Amos Beimel, Kobbi Nissim, and Uri Stemmer. Private learning and sanitization: Pure
vs. approximate differential privacy. In Approximation, Randomization, and Combinatorial Optimization. Algorithms and Techniques, pages 363–378. Springer, 2013.

(they called it Semi-Private Learning...)

- Papernot, N., Abadi, M., Erlingsson, U., Goodfellow, I., & Talwar, K. (2017). Semi-supervised knowledge transfer for deep learning from private training data. In ICLR-17.

The idea of integrating out the noise by leveraging the known noise structure were considered in:

- Williams, O., & McSherry, F. (2010). Probabilistic inference and differential privacy. Advances in Neural Information Processing Systems, 23, 2451-2459.

- Balle, B., & Wang, Y. X. (2018). Improving the Gaussian Mechanism for Differential Privacy: Analytical Calibration and Optimal Denoising. In International Conference on Machine Learning (pp. 394-403).

And many subsequent work.

The contribution of this work is in combining these known pieces (without citing some of the earlier work) to achieve a reasonably strong set of experimental results (for LDP standard).  I believe this is the first experimental study that uses VAE for the dimension reduction, however, this alone is not sufficient to carry the paper in my opinion; especially since the setting is now much easier, with access to a public dataset.

The reviewers question the experiments are baselines are usually not using a public dataset as well as the practicality of the proposed method.   Also, connections to some of the existing work on private data release (a.k.a., private synthetic data generation) were note clarified. For these reasons, there were not sufficient support among the reviewers to push the paper through. 

The authors are encouraged to revise the paper according to the suggestions and resubmit in the next appropriate venue.
Decision: Reject