Learning to Noise: Application-Agnostic Data Sharing with Local Differential Privacy

1 INTRODUCTION . The collection of personal data is ubiquitous , and unavoidable for many in everyday life . While this has undeniably improved the quality and user experience of many products and services , evidence of data misuse and data breaches ( Sweeney , 1997 ; Jolly , 2020 ) have brought the concept of data privacy into sharp focus , fueling both regulatory changes as well as a shift in personal preferences . The onus has now fallen on organizations to determine if they are willing and able to collect personal data under these changing expectations . There is thus a growing need to collect data in a privacypreserving manner , that can still be used to improve products and services . Often coined the ‘ gold standard ’ of privacy guarantees , central differential privacy ( CDP ) ( Dwork & Roth , 2014 ) protects against an adversary determining the presence of a user in a dataset . It provides a quantifiable definition , is robust to post-processing , and allows for the protection of groups of people via its composability property ( Dwork et al. , 2006 ; Dwork & Roth , 2014 ) . The CDP framework relies on the addition of noise to the output of statistical queries on a dataset , in such a way that the same information can be extracted from that dataset whether or not any given individual is present . One can train machine learning models such that the model is CDP with respect to the training set by using training methods such as DP-SGD ( Abadi et al. , 2016 ) , DP-Adam ( Gylberth et al. , 2017 ) or PATE ( Papernot et al. , 2017 ) . Given access to a clean labelled dataset , one could train , for example , a CDP classifier this way . Similarly , one could ‘ share ’ data privately by training a generative model , such as a variational autoencoder ( VAE ) or generative adversarial network ( GAN ) , with a CDP training method , and then construct a synthetic dataset satisfying CDP with respect to the training set by generating samples from this model ( Xie et al. , 2018 ; Triastcyn & Faltings , 2019 ; Acs et al. , 2019 ; Takagi et al. , 2021 ) . While some of these approaches require only an unlabelled training set , their applications are limited in that they generate only synthetic samples that are likely under the original training set distribution . Firstly , changes in the data distribution warrant the re-training of the generative model . Secondly , the synthetic points are only representative samples , and so we lack any information about the features of given individuals . Clearly , this limits the range of applications : the model can not be used to join private and clean datasets , for applications in which we expect a distributional shift , or for privately collecting new data . Furthermore , CDP also requires a trustworthy database administrator . Federated learning ( McMahan et al. , 2017 ; Agarwal et al. , 2018 ; Rodrı́guez-Barroso et al. , 2020 ) is a related technique in which a model can be trained privately , without placing trust in a database administrator . The model is trained across multiple decentralized devices , omitting the need to share data with a central server . Unfortunately , by construction , federated learning does not allow for any data to be privately shared or collected , and is only concerned with the privacy of the model . An approach that allows data collection , while protecting the privacy of an individual against even the database administrator , is to construct a mechanism that privatizes the features of a given individual locally , before collection . Warner ( 1965 ) developed a mechanism , known as randomized response , to preserve the privacy of survey respondents : when answering a sensitive ( binary ) question , the respondent is granted plausible deniability by giving a truthful answer if a fair coin flip returns heads , and answering yes otherwise . Recent work has further developed this idea , often referred to as local differential privacy ( LDP ) ( Kasiviswanathan et al. , 2008 ; Duchi et al. , 2013 ) . LDP provides a mathematically-provable privacy guarantee for members of a database against both adversaries and database administrators . Many existing LDP mechanism focus on the collection of low-dimensional data like summary statistics , or on mechanisms which do not easily generalize to different data types . Erlingsson et al . ( 2014 ) , Ding et al . ( 2017 ) and Tang et al . ( 2017 ) introduce methods for collecting such data repeatedly over time . Erlingsson et al . ( 2014 ) find that for one time collection , directly noising data ( after hashing to a bloom filter ) is their best approach for inducing privacy . Ren et al . ( 2018 ) extend this bloom filter based approach , and attempt to estimate the clean distribution of their LDP data in order to generate a LDP synthetic dataset , though we note that the range of applications is limited , as with CDP synthetic generation above . In this paper , we adapt well-established techniques from representation learning to address the fundamental limitation of LDP in the context of high-dimensional data : datapoints in high dimensional space require prohibitive levels of noise to locally privatize ( the privacy budget , , naively scales linearly with the dimensionality ) . To motivate our approach , consider the fact that it is often a good approximation to assume that a given high-dimensional dataset lives on a much lower dimensional manifold . Applying a general privatization mechanism to low-dimensional representations should thus enable us to learn how to add noise to the high-dimensional data efficiently and applicationagnostically . Our approach is inspired by the VAE ( Kingma & Welling , 2014 ; Rezende et al. , 2014 ) ; we demonstrate that sampling in latent space is equivalent to passing a datapoint through an LDP Laplace mechanism . Furthermore , reconstructing a datapoint is equivalent to adding a complex noise distribution to the raw features , thereby inducing LDP . Our randomized algorithm , which we refer to as the variational Laplace mechanism ( VLM ) , satisfies the strict guarantees of LDP , and is agnostic to both data type and downstream task . We demonstrate that we can use the data privatized with our mechanism to train downstream machine learning models that act on both clean and privatized data at inference time . Furthermore , we demonstrate multiple concrete applications of our model : we privately collect data from individuals for downstream model training ; we use a transfer-learning-inspired approach to privately collect data of an unseen class type upon which we train a classifier ; and we augment a clean dataset with additional privatized features to improve the accuracy of a classifier on the combined data . None of these applications can be solved with CDP , and we find significant performance gains over the naive approach of directly noising the data . 2 BASIC DEFINITIONS AND NOTATION . To formalize the concept of differential privacy , we first introduce some definitions and notation . Definition ( ( , δ ) -central differential privacy ) : Let A : D → Z be a randomized algorithm , that takes as input datasets from the dataset domain D. We sayA is ( , δ ) -central differentially private if for , δ ≥ 0 , for all subsets S ⊆ Z , and for all neighboring datasets D , D′ ∈ D , we have p ( A ( D ) ∈ S ) ≤ exp ( ) p ( A ( D′ ) ∈ S ) + δ ( 1 ) where for D and D′ to be neighboring means that they are identical in all but one datapoint . Intuitively , this states that one can not tell ( with a level of certainty determined by ( , δ ) ) whether an individual is present in a database or not . When δ = 0 we say A satisfies -CDP . Definition ( ` 1 sensitivity ) : The ` 1 sensitivity of a function f : D → Rk is defined as ∆f = max adjacent ( D , D′ ) ||f ( D ) − f ( D′ ) ||1 ( 2 ) where adjacent ( D , D′ ) implies D , D′ ∈ D are neighboring datasets . Definition ( Laplace mechanism ) : The Laplace mechanismM ( central ) : D → Rk is a randomized algorithm defined as M ( central ) ( D , f ( · ) , ) = f ( D ) + ( s1 , . . . , sk ) ( 3 ) for D ∈ D , si ∼ Laplace ( 0 , ∆f/ ) , and some transformation function f : D → Rk . The Laplace mechanism induces -CDP ; see Dwork & Roth ( 2014 ) for proof . While CDP relies on a trusted database administrator , LDP provides a much stricter guarantee in which the individual does not need to trust an administrator . Instead individuals are able to privatize their data before sending it using a local randomized algorithm . Definition ( ( , δ ) -local differential privacy ) : A local randomized algorithmA : X → Z , that takes as input a datapoint from the data domain X , satisfies ( , δ ) -local differential privacy if for , δ ≥ 0 , for all S ⊆ Z , and for any inputs x , x′ ∈ X , p ( A ( x ) ∈ S ) ≤ exp ( ) p ( A ( x′ ) ∈ S ) + δ ( 4 ) When δ = 0 we say A satisfies -LDP . Definition ( Local Laplace mechanism ) : The local Laplace mechanism M ( local ) : X → Rk is a randomized algorithm defined as M ( local ) ( x , f ( · ) , ) = f ( x ) + ( s1 , . . . , sk ) ( 5 ) for x ∈ X , si ∼ Laplace ( 0 , ∆f/ ) , and some transformation function f : X → Z , where Z ⊆ Rk and the ` 1 sensitivity of f ( · ) is defined as ∆f = maxx , x′∈X ||f ( x ) − f ( x′ ) ||1 . The local Laplace mechanism satisfies -LDP ( see Appendix A for proof ) . Another common choice of mechanism for privatizing continuous data is the Gaussian mechanism , which satisfies ( , δ > 0 ) -LDP . For the remainder of the paper however , we exclusively study the local Laplace mechanism since it provides a strong privacy guarantee ( i.e . δ = 0 ) . We note that our approach could be used to learn a Gaussian mechanism with minimal modification . 3 PROPOSED METHOD . Early work on data collection algorithms , such as randomized response , have relied on very simple randomized algorithms to induce privacy . Throughout this paper we have benchmarked our results against such a mechanism , in which we add Laplace noise to all continuous features , and flip each of the categorical features with some probability . By the composition theorem ( Dwork & Roth , 2014 ) , each feature then contributes towards the overall LDP guarantee of the d-dimensional datapoint as = ∑d i=1 i . Since we have no prior knowledge about which features are most important , we choose the Laplace noise level ( or flip probability ) for each feature to be such that i = /d . See Appendix E.2 for further details . As with our approach , this benchmark mechanism can act on any date type , and forms a downstream task agnostic LDP version of the data . As d increases , i decreases for each feature i . The noise required to induce -LDP thus grows with data dimensionality . For high-dimensional datasets like images or large tables , features are often highly correlated ; consequently , noising features independently is wasteful towards privatizing the information content in each datapoint . A more effective approach to privatization involves noising a learned lower-dimensional representation of each datapoint using a generic noising mechanism . To this end , we use a VAE-based approach to learn a low-dimensional latent representation of our data . This learned mapping from data space to latent space forms our function f ( · ) in Equation 5 , and requires only a small unlabelled dataset from a similar distribution , which most organizations will typically already have access to , either internally or publicly . Applying the Laplace mechanism ( as described in Section 3.1 ) thus ensures the encoded latents , as well as reconstructed datapoints , satisfy LDP . We can therefore privatize data at the latent level or the original-feature level ; preference between these two options is application specific , and we investigate both experimentally . Data owners can apply this learned LDP mechanism to their data before sharing , and the database administrator forms an LDP data set from the collected data . This set , along with information on the type of noise added , is used to train downstream machine learning algorithms . Though our privatization method is task agnostic , in this paper we focus on classification tasks in which we have some features x for which we want to predict the corresponding label y . At inference time , we show that this classifier can act on either clean or privatized datapoints , depending on the application .

Summary: For LDP, when applying noise directly to high-dimensional data, the required noise entirely destroys data utility. In this paper, authors introduce a novel, application-agnostic privatization mechanism that leverages representation learning to overcome the prohibitive noise requirements of direct methods. They further demonstrate that this privatization mechanism can be used to train machine learning algorithms across a range of applications. They achieve significant gains in performance for high-dimensional data.