Improving Differentially Private Models with Active Learning

1 INTRODUCTION . Privacy concerns surfaced with the increased adoption of machine learning in domains like healthcare ( LeCun et al. , 2015 ) . One widely adopted framework for measuring privacy characteristics of randomized algorithms , such as machine learning techniques , is differential privacy ( Dwork et al. , 2006 ) . Abadi et al . ( 2016 ) introduced an algorithm for differentially private stochastic gradient descent ( DP-SGD ) , which made it feasible to scale differential privacy guarantees to neural networks . DP-SGD is now the de facto algorithm used for training neural networks with privacy guarantees . There is , however , a crucial problem for models with a large number of parameters : it is difficult to achieve both non-trivial privacy guarantees and good accuracy . The reasons for this are numerous and involved , but the basic intuition is that DP-SGD involves clipping gradients and then adding noise to those gradients . However , gradient clipping has an increasing relative effect as the number of model parameters grows . This reduces the applicability of differential privacy to deep learning in practice , where strong performance tends to require a large number of model parameters.1 In this work , we propose a method to mitigate this problem , making DP learning feasible for modern image classifiers . The method is based on the observation that each DP-SGD update consumes privacy budget in proportion to the quotient of the batch size and the training-set size . Thus , by increasing the number of effective training examples , we can improve accuracy while maintaining the same privacy guarantees . Since obtaining more private training samples will generally be nontrivial , we focus on using ‘ public ’ data to augment the training set . This involves assuming that there will be unlabeled2 public data that is sufficiently related to our private data to be useful , but we think this is a reasonable assumption to make , for two reasons : First , it is an assumption made by most of the semi-supervised learning literature ( Oliver et al. , 2018 ) . Second , one of our techniques will explicitly address the situation where the ‘ sufficiently related ’ clause partially breaks down . In summary , our contributions are : • We introduce an algorithm ( DIVERSEPUBLIC ) to select diverse representative samples from a public dataset to fine-tune a DP classifier . 1For example , the best performing CIFAR-10 classifier from Zagoruyko & Komodakis ( 2016 ) has 32.5 million parameters , while the private baseline we describe in Section 4.1 has about 26 thousand . 2 If there is labeled public data , the situation is even better , but this is likely to be rare and is not the setting we consider here . • We then describe another algorithm ( NEARPRIVATE ) that pays a privacy cost to reference the private training data when querying the public dataset for representative samples . • We establish new state-of-the-art results on the MNIST and SVHN datasets by fine-tuning DP models using simple active learning techniques , and then improve upon those results further using DIVERSEPUBLIC and NEARPRIVATE . • We open source all of our experimental code.3 2 BACKGROUND . 2.1 DIFFERENTIAL PRIVACY . We reason about privacy within the framework of differential privacy ( Dwork et al. , 2006 ) . In this paper , the random algorithm we analyze is the training algorithm , and guarantees are measured with respect to the training data . Informally , an algorithm is said to be differentially private if its behavior is indistinguishable on pairs of datasets that only differ by one point . That is , an observer can not tell whether a particular point was included in the model ’ s training set simply by observing the output of the training algorithm . Formally , for a training algorithm A to be ( ε , δ ) -differentially private , we require that it satisfies the following property for all pairs of datasets ( d , d′ ) differing in exactly one data point and all possible subsets S ⊂ Range ( A ) : Pr [ A ( d ) ∈ S ] ≤ eεPr [ A ( d′ ) ∈ S ] + δ where δ is a ( small ) probability for which we tolerate the property not to be satisfied . The parameter ε measures the strength of the privacy guarantee : the smaller the value of ε is , the stronger the privacy guarantee is . This guarantee is such that the output of a differentially private algorithm can be post-processed at no impact to the strength of the guarantee provided . In case privacy needs to be defined at a different granularity than invidividual training points , the guarantee degrades by a factor which , naively , is the number of points that are included in the granularity considered . See Dwork et al . ( 2014a ) for further information . 2.2 DIFFERENTIALLY PRIVATE STOCHASTIC GRADIENT DESCENT ( DP-SGD ) . Building on earlier work ( Chaudhuri et al. , 2011 ; Song et al. , 2013 ) , Abadi et al . ( 2016 ) introduce a variant of stochastic gradient descent to train deep neural networks with differential privacy guarantees . Two modifications are made to vanilla SGD . First , gradients are computed on a per-example basis and clipped to have a maximum known ` 2 norm . This bounds the sensitivity of the training procedure with respect to each individual training point . Second , noise calibrated to have a standard deviation proportional to this sensitivity is added to the average gradient . This results in a training algorithm known as differentially private SGD ( DP-SGD ) . Unfortunately ( as discussed in Section 1 ) , DP-SGD does not perform well for models with large parameter counts , which motivates the improvements proposed in the next section . 3 IMPROVING DIFFERENTIALLY PRIVATE MODELS WITH ACTIVE LEARNING . This paper introduces the following high-level process to improve the performance of an existing DP classifier : First , find a public insensitive dataset containing relevant samples . Second , carefully select a subset of public samples that can improve the performance of the classifier . Third , fine-tune the existing classifier on the selected samples with labels . We want to perform the selection and the fine-tuning in a way which does not compromise the privacy guarantees of the existing classifier . The first step can be done using domain knowledge about the classifier , e.g. , utilizing relevant public genomic datasets for a DP classifier of genomics data . We assume standard fine-tuning techniques for the last step . Therefore , the problem boils down to efficiently selecting samples from the public dataset while preserving privacy . We also assume a limit on the number of selected samples . This 3URL blinded for anonymity . Algorithm 1 The DIVERSEPUBLIC Algorithm Input : Mdp with privacy cost εdp , Dpublic , Nlabeled , k > Nlabeled , p , Ncluster , Neach Output : M∗dp fine-tuned on selected public data with the same privacy cost εdp Epublic ←Mdp ( Dpublic ) [ −1 ] . Compute ‘ embeddings ’ of public data P ← PCA ( Epublic , p ) . Perform PCA on embeddings and get first p PCs Spublic ← P × SelectUncertain ( Dpublic , k ) . Project k most ‘ uncertain ’ public data onto PCs ClusterCenters← K-Means ( Spublic ) . Cluster projected embeddings to get Ncluster clusters Dlabeled ← ∅ for i = 1 to Ncluster do . Label Neach data points from each cluster Dlabeled = Dlabeled ∪ TakeCenterPoints ( ClusterCenters ) end for M∗dp ← FineTune ( Mdp , Dlabeled ) limit is relevant when the samples are unlabeled , in which case it controls the cost of labeling ( e.g. , hiring human annotators to process the selected samples ) . We introduce two active learning algorithms , DIVERSEPUBLIC and NEARPRIVATE , for sampleselection . These algorithms make different assumptions about access to the private training data but are otherwise drop-in replacements in the end-to-end process . 3.1 PROBLEM STATEMENT AND BASELINE METHODS . We are given a differentially private modelMdp trained and tested on private sensitive data Dtrain and Dtest respectively . The privacy cost of trainingMdp on Dtrain is εdp ( we omit δ for brevity in this paper , but it composes similarly ) . And we have an extra set of public insensitive unlabeled data Dpublic which can be utilized to further improveMdp . Given the number of extra data Nlabeled that we are allowed to request labels for , and the total privacy budget εlimit of the improved modelM∗dp , we want to efficiently pick Dlabeled ⊂ Dpublic where |Dlabeled| = Nlabeled , using which we can fine-tune Mdp toM∗dp of better performance on Dtest . The simplest baseline method we will use is to choose Nlabeled ‘ random ’ samples out of Dpublic for fine-tuning . The other baseline that we use is Uncertain Sampling ( Settles , 2009 ) , according to either the ‘ entropy ’ of the logits or the ‘ difference ’ between the two largest logits . These are widely used active learning methods and serve as strong baselines . 3.2 THE DIVERSEPUBLIC METHOD . The baseline methods may select many redundant examples to label . To efficiently select diverse representative samples , we propose the DIVERSEPUBLIC method , adapted from clustering based active learning methods ( Nguyen & Smeulders , 2004 ) . Given a DP modelMdp of cost εdp , we obtain the ‘ embeddings ’ ( the activations before the logits ) Epublic of all Dpublic , and perform PCA on Epublic as is done in Raghu et al . ( 2017 ) . Then we select a number ( more than Nlabeled ) of uncertain points ( according to , e.g. , the logit entropy of the private model ) from Dpublic , project their embeddings onto the top few principal components , and cluster those projections into groups . Finally , we pick a number of samples from each representative group up to Nlabeled in total and fine-tune the DP model with these Dlabeled . Though this procedure accessesMdp , it adds nothing to εdp , since there is no private data referenced and the output of DP models can be post-processed at no additional privacy cost . It can be applied even to models for which we can not access the original training data.4 Algorithm 1 presents more details of the DIVERSEPUBLIC method . 3.3 THE NEARPRIVATE METHOD . The DIVERSEPUBLIC method works well under the assumption that Dpublic has the same distribution as Dtrain . However , this may not be a reasonable assumption in general ( Oliver et al. , 2018 ) . For instance , there may be a subset of Dpublic about which our pre-trained model has high uncertainty but which can not improve performance if sampled . This may be because that subset contains corrupted data or it may be due simply to distribution shift . In order to mitigate this issue , we propose to 4e.g. , models published to a repository like TensorFlow Hub : https : //www.tensorflow.org/hub Algorithm 2 The NEARPRIVATE Algorithm Input : Mdp with privacy cost εdp , Dpublic , Nlabeled , k > Nlabeled , p , Dtrain , εdpPCA and εsupport Output : M∗dp fine-tuned on selected public data with privacy cost εdp + εdpPCA + εsupport Etrain ←Mdp ( Dtrain ) [ −1 ] . Compute ‘ embeddings ’ of private data P ← DP-PCA ( Etrain , εdpPCA , p ) . Perform DP-PCA on embeddings and get first p PCs Strain ← P × SelectUncertain ( Dtrain , k ) . Project k most ‘ uncertain ’ private data onto PCs Spublic ← P × SelectUncertain ( Dpublic , k ) . Project k most ‘ uncertain ’ public data onto PCs NeighborCounts← ∅ for a ∈ Strain do . In PC-space , assign each private point to exactly one public point Increment ( NeighborCounts , NearestNeighbor ( a , Spublic ) ) end for for b ∈ Spublic do . Compute support with Laplacian noise for each public data point Nsupport ( b ) ← NeighborCounts ( b ) + LaplaceNoise ( 1/εsupport ) end for Dlabeled ← TakeArgmaxPoints ( Nsupport ) . Label Nlabeled data points of the highest Nsupport M∗dp ← FineTune ( Mdp , Dlabeled ) check query points against our private data and decline to label query points that are too far from any training points ( in projected-embedding-space ) . But doing this check while preserving privacy guarantees is nontrivial , since it involves processing the private training data itself in addition to Mdp . Given a DP model of cost εdp , we obtain the embeddings Etrain of all Dtrain , and perform differentially private PCA ( DP-PCA ) ( Dwork et al. , 2014b ) on Etrain at a privacy cost of εdpPCA . We select a number of uncertain points from both Dtrain and Dpublic . Then in the space of low dimensional DP-PCA projections , we assign each uncertain private example to exactly one uncertain public example according to Euclidean distance . This yields , for each uncertain public example , a count Nsupport of ‘ nearby ’ uncertain private examples . Finally , we choose Nlabeled points with the largest values ofNsupport+LaplaceNoise ( 1/εsupport ) from the uncertain public data . This sampling procedure is differentially private with cost εsupport due to the Laplace mechanism ( Dwork et al. , 2014a ) . The total privacy cost of NEARPRIVATE is composed of the budgets expended to perform each of the three operations that depend on the private data : εdp + εdpPCA + εsupport . More details in Algorithm 2 .

Summary: This paper considers the problem of how to improve the performance of an existing DP classifier, with the help of the labelled public data. The paper considers the following process: 1. find a public dataset containing relevant samples; 2. carefully select a subset of public samples that can improve the performance of the classifier; 3 fine-tune the existing classifier on the selected samples. Two different techniques from active learning are utilized in order to select representative samples from a public dataset and fine-tune a DP classifier. This paper also conducts some experiments on   the MNIST and SVHN datasets and demonstrates improvement compared with the benchmark.