Learning to Noise: Application-Agnostic Data Sharing with Local Differential Privacy

1 INTRODUCTION . The collection of personal data is ubiquitous , and unavoidable for many in everyday life . While this has undeniably improved the quality and user experience of many products and services , evidence of data misuse and data breaches ( Sweeney , 1997 ; Jolly , 2020 ) have brought the concept of data privacy into sharp focus , fueling both regulatory changes as well as a shift in personal preferences . The onus has now fallen on organizations to determine if they are willing and able to collect personal data under these changing expectations . There is thus a growing need to collect data in a privacypreserving manner , that can still be used to improve products and services . Often coined the ‘ gold standard ’ of privacy guarantees , central differential privacy ( CDP ) ( Dwork & Roth , 2014 ) protects against an adversary determining the presence of a user in a dataset . It provides a quantifiable definition , is robust to post-processing , and allows for the protection of groups of people via its composability property ( Dwork et al. , 2006 ; Dwork & Roth , 2014 ) . The CDP framework relies on the addition of noise to the output of statistical queries on a dataset , in such a way that the same information can be extracted from that dataset whether or not any given individual is present . One can train machine learning models such that the model is CDP with respect to the training set by using training methods such as DP-SGD ( Abadi et al. , 2016 ) , DP-Adam ( Gylberth et al. , 2017 ) or PATE ( Papernot et al. , 2017 ) . Given access to a clean labelled dataset , one could train , for example , a CDP classifier this way . Similarly , one could ‘ share ’ data privately by training a generative model , such as a variational autoencoder ( VAE ) or generative adversarial network ( GAN ) , with a CDP training method , and then construct a synthetic dataset satisfying CDP with respect to the training set by generating samples from this model ( Xie et al. , 2018 ; Triastcyn & Faltings , 2019 ; Acs et al. , 2019 ; Takagi et al. , 2021 ) . While some of these approaches require only an unlabelled training set , their applications are limited in that they generate only synthetic samples that are likely under the original training set distribution . Firstly , changes in the data distribution warrant the re-training of the generative model . Secondly , the synthetic points are only representative samples , and so we lack any information about the features of given individuals . Clearly , this limits the range of applications : the model can not be used to join private and clean datasets , for applications in which we expect a distributional shift , or for privately collecting new data . Furthermore , CDP also requires a trustworthy database administrator . Federated learning ( McMahan et al. , 2017 ; Agarwal et al. , 2018 ; Rodrı́guez-Barroso et al. , 2020 ) is a related technique in which a model can be trained privately , without placing trust in a database administrator . The model is trained across multiple decentralized devices , omitting the need to share data with a central server . Unfortunately , by construction , federated learning does not allow for any data to be privately shared or collected , and is only concerned with the privacy of the model . An approach that allows data collection , while protecting the privacy of an individual against even the database administrator , is to construct a mechanism that privatizes the features of a given individual locally , before collection . Warner ( 1965 ) developed a mechanism , known as randomized response , to preserve the privacy of survey respondents : when answering a sensitive ( binary ) question , the respondent is granted plausible deniability by giving a truthful answer if a fair coin flip returns heads , and answering yes otherwise . Recent work has further developed this idea , often referred to as local differential privacy ( LDP ) ( Kasiviswanathan et al. , 2008 ; Duchi et al. , 2013 ) . LDP provides a mathematically-provable privacy guarantee for members of a database against both adversaries and database administrators . Many existing LDP mechanism focus on the collection of low-dimensional data like summary statistics , or on mechanisms which do not easily generalize to different data types . Erlingsson et al . ( 2014 ) , Ding et al . ( 2017 ) and Tang et al . ( 2017 ) introduce methods for collecting such data repeatedly over time . Erlingsson et al . ( 2014 ) find that for one time collection , directly noising data ( after hashing to a bloom filter ) is their best approach for inducing privacy . Ren et al . ( 2018 ) extend this bloom filter based approach , and attempt to estimate the clean distribution of their LDP data in order to generate a LDP synthetic dataset , though we note that the range of applications is limited , as with CDP synthetic generation above . In this paper , we adapt well-established techniques from representation learning to address the fundamental limitation of LDP in the context of high-dimensional data : datapoints in high dimensional space require prohibitive levels of noise to locally privatize ( the privacy budget , , naively scales linearly with the dimensionality ) . To motivate our approach , consider the fact that it is often a good approximation to assume that a given high-dimensional dataset lives on a much lower dimensional manifold . Applying a general privatization mechanism to low-dimensional representations should thus enable us to learn how to add noise to the high-dimensional data efficiently and applicationagnostically . Our approach is inspired by the VAE ( Kingma & Welling , 2014 ; Rezende et al. , 2014 ) ; we demonstrate that sampling in latent space is equivalent to passing a datapoint through an LDP Laplace mechanism . Furthermore , reconstructing a datapoint is equivalent to adding a complex noise distribution to the raw features , thereby inducing LDP . Our randomized algorithm , which we refer to as the variational Laplace mechanism ( VLM ) , satisfies the strict guarantees of LDP , and is agnostic to both data type and downstream task . We demonstrate that we can use the data privatized with our mechanism to train downstream machine learning models that act on both clean and privatized data at inference time . Furthermore , we demonstrate multiple concrete applications of our model : we privately collect data from individuals for downstream model training ; we use a transfer-learning-inspired approach to privately collect data of an unseen class type upon which we train a classifier ; and we augment a clean dataset with additional privatized features to improve the accuracy of a classifier on the combined data . None of these applications can be solved with CDP , and we find significant performance gains over the naive approach of directly noising the data . 2 BASIC DEFINITIONS AND NOTATION . To formalize the concept of differential privacy , we first introduce some definitions and notation . Definition ( ( , δ ) -central differential privacy ) : Let A : D → Z be a randomized algorithm , that takes as input datasets from the dataset domain D. We sayA is ( , δ ) -central differentially private if for , δ ≥ 0 , for all subsets S ⊆ Z , and for all neighboring datasets D , D′ ∈ D , we have p ( A ( D ) ∈ S ) ≤ exp ( ) p ( A ( D′ ) ∈ S ) + δ ( 1 ) where for D and D′ to be neighboring means that they are identical in all but one datapoint . Intuitively , this states that one can not tell ( with a level of certainty determined by ( , δ ) ) whether an individual is present in a database or not . When δ = 0 we say A satisfies -CDP . Definition ( ` 1 sensitivity ) : The ` 1 sensitivity of a function f : D → Rk is defined as ∆f = max adjacent ( D , D′ ) ||f ( D ) − f ( D′ ) ||1 ( 2 ) where adjacent ( D , D′ ) implies D , D′ ∈ D are neighboring datasets . Definition ( Laplace mechanism ) : The Laplace mechanismM ( central ) : D → Rk is a randomized algorithm defined as M ( central ) ( D , f ( · ) , ) = f ( D ) + ( s1 , . . . , sk ) ( 3 ) for D ∈ D , si ∼ Laplace ( 0 , ∆f/ ) , and some transformation function f : D → Rk . The Laplace mechanism induces -CDP ; see Dwork & Roth ( 2014 ) for proof . While CDP relies on a trusted database administrator , LDP provides a much stricter guarantee in which the individual does not need to trust an administrator . Instead individuals are able to privatize their data before sending it using a local randomized algorithm . Definition ( ( , δ ) -local differential privacy ) : A local randomized algorithmA : X → Z , that takes as input a datapoint from the data domain X , satisfies ( , δ ) -local differential privacy if for , δ ≥ 0 , for all S ⊆ Z , and for any inputs x , x′ ∈ X , p ( A ( x ) ∈ S ) ≤ exp ( ) p ( A ( x′ ) ∈ S ) + δ ( 4 ) When δ = 0 we say A satisfies -LDP . Definition ( Local Laplace mechanism ) : The local Laplace mechanism M ( local ) : X → Rk is a randomized algorithm defined as M ( local ) ( x , f ( · ) , ) = f ( x ) + ( s1 , . . . , sk ) ( 5 ) for x ∈ X , si ∼ Laplace ( 0 , ∆f/ ) , and some transformation function f : X → Z , where Z ⊆ Rk and the ` 1 sensitivity of f ( · ) is defined as ∆f = maxx , x′∈X ||f ( x ) − f ( x′ ) ||1 . The local Laplace mechanism satisfies -LDP ( see Appendix A for proof ) . Another common choice of mechanism for privatizing continuous data is the Gaussian mechanism , which satisfies ( , δ > 0 ) -LDP . For the remainder of the paper however , we exclusively study the local Laplace mechanism since it provides a strong privacy guarantee ( i.e . δ = 0 ) . We note that our approach could be used to learn a Gaussian mechanism with minimal modification . 3 PROPOSED METHOD . Early work on data collection algorithms , such as randomized response , have relied on very simple randomized algorithms to induce privacy . Throughout this paper we have benchmarked our results against such a mechanism , in which we add Laplace noise to all continuous features , and flip each of the categorical features with some probability . By the composition theorem ( Dwork & Roth , 2014 ) , each feature then contributes towards the overall LDP guarantee of the d-dimensional datapoint as = ∑d i=1 i . Since we have no prior knowledge about which features are most important , we choose the Laplace noise level ( or flip probability ) for each feature to be such that i = /d . See Appendix E.2 for further details . As with our approach , this benchmark mechanism can act on any date type , and forms a downstream task agnostic LDP version of the data . As d increases , i decreases for each feature i . The noise required to induce -LDP thus grows with data dimensionality . For high-dimensional datasets like images or large tables , features are often highly correlated ; consequently , noising features independently is wasteful towards privatizing the information content in each datapoint . A more effective approach to privatization involves noising a learned lower-dimensional representation of each datapoint using a generic noising mechanism . To this end , we use a VAE-based approach to learn a low-dimensional latent representation of our data . This learned mapping from data space to latent space forms our function f ( · ) in Equation 5 , and requires only a small unlabelled dataset from a similar distribution , which most organizations will typically already have access to , either internally or publicly . Applying the Laplace mechanism ( as described in Section 3.1 ) thus ensures the encoded latents , as well as reconstructed datapoints , satisfy LDP . We can therefore privatize data at the latent level or the original-feature level ; preference between these two options is application specific , and we investigate both experimentally . Data owners can apply this learned LDP mechanism to their data before sharing , and the database administrator forms an LDP data set from the collected data . This set , along with information on the type of noise added , is used to train downstream machine learning algorithms . Though our privatization method is task agnostic , in this paper we focus on classification tasks in which we have some features x for which we want to predict the corresponding label y . At inference time , we show that this classifier can act on either clean or privatized datapoints , depending on the application .

Summary: In this paper, the authors present a generative-model-based Laplace mechanism. By training the VAE on some dataset, the trained encoder can be used to privatize raw data towards epsilon, delta-LDP. Though the method is novel, the privacy guarantee of the proposed method is not clearly stated and proved. Related experiments are not convincing, either.