Title: IMPROVING DIFFERENTIALLY PRIVATE MODELS WITH ACTIVE LEARNING

Abstract: Broad adoption of machine learning techniques has increased privacy concerns for models trained on sensitive data such as medical records. Existing techniques for training differentially private (DP) models give rigorous privacy guarantees, but applying these techniques to neural networks can severely degrade model performance. This performance reduction is an obstacle to deploying private models in the real world. In this work, we improve the performance of DP models by fine-tuning them through active learning on public data. We introduce two new techniques -DIVERSEPUBLIC and NEARPRIVATE -for doing this fine-tuning in a privacy-aware way. For the MNIST and SVHN datasets, these techniques improve state-of-the-art accuracy for DP models while retaining privacy guarantees.

1. INTRODUCTION:

Privacy concerns surfaced with the increased adoption of machine learning in domains like healthcare (LeCun et al., 2015) . One widely adopted framework for measuring privacy characteristics of randomized algorithms, such as machine learning techniques, is differential privacy (Dwork et al., 2006) . Abadi et al. (2016) introduced an algorithm for differentially private stochastic gradient descent (DP-SGD), which made it feasible to scale differential privacy guarantees to neural networks. DP-SGD is now the de facto algorithm used for training neural networks with privacy guarantees.

There is, however, a crucial problem for models with a large number of parameters: it is difficult to achieve both non-trivial privacy guarantees and good accuracy. The reasons for this are numerous and involved, but the basic intuition is that DP-SGD involves clipping gradients and then adding noise to those gradients. However, gradient clipping has an increasing relative effect as the number of model parameters grows. This reduces the applicability of differential privacy to deep learning in practice, where strong performance tends to require a large number of model parameters. 1In this work, we propose a method to mitigate this problem, making DP learning feasible for modern image classifiers. The method is based on the observation that each DP-SGD update consumes privacy budget in proportion to the quotient of the batch size and the training-set size. Thus, by increasing the number of effective training examples, we can improve accuracy while maintaining the same privacy guarantees. Since obtaining more private training samples will generally be nontrivial, we focus on using 'public' data to augment the training set. This involves assuming that there will be unlabeled 2 public data that is sufficiently related to our private data to be useful, but we think this is a reasonable assumption to make, for two reasons: First, it is an assumption made by most of the semi-supervised learning literature (Oliver et al., 2018) . Second, one of our techniques will explicitly address the situation where the 'sufficiently related' clause partially breaks down.

In summary, our contributions are:

• We introduce an algorithm (DIVERSEPUBLIC ) to select diverse representative samples from a public dataset to fine-tune a DP classifier.

• We then describe another algorithm (NEARPRIVATE ) that pays a privacy cost to reference the private training data when querying the public dataset for representative samples. • We establish new state-of-the-art results on the MNIST and SVHN datasets by fine-tuning DP models using simple active learning techniques, and then improve upon those results further using DIVERSEPUBLIC and NEARPRIVATE . • We open source all of our experimental code.3 

2. BACKGROUND:



2.1. DIFFERENTIAL PRIVACY:

We reason about privacy within the framework of differential privacy (Dwork et al., 2006) . In this paper, the random algorithm we analyze is the training algorithm, and guarantees are measured with respect to the training data.

Informally, an algorithm is said to be differentially private if its behavior is indistinguishable on pairs of datasets that only differ by one point. That is, an observer cannot tell whether a particular point was included in the model's training set simply by observing the output of the training algorithm. Formally, for a training algorithm A to be (ε, δ)-differentially private, we require that it satisfies the following property for all pairs of datasets (d, d ) differing in exactly one data point and all possible subsets S ⊂ Range(A):

P r[A(d) ∈ S] ≤ e ε P r[A(d ) ∈ S] + δ

where δ is a (small) probability for which we tolerate the property not to be satisfied. The parameter ε measures the strength of the privacy guarantee: the smaller the value of ε is, the stronger the privacy guarantee is.

This guarantee is such that the output of a differentially private algorithm can be post-processed at no impact to the strength of the guarantee provided. In case privacy needs to be defined at a different granularity than invidividual training points, the guarantee degrades by a factor which, naively, is the number of points that are included in the granularity considered. See Dwork et al. (2014a) for further information.

2.2. DIFFERENTIALLY PRIVATE STOCHASTIC GRADIENT DESCENT (DP-SGD):

Building on earlier work (Chaudhuri et al., 2011; Song et al., 2013) , Abadi et al. (2016) introduce a variant of stochastic gradient descent to train deep neural networks with differential privacy guarantees. Two modifications are made to vanilla SGD. First, gradients are computed on a per-example basis and clipped to have a maximum known 2 norm. This bounds the sensitivity of the training procedure with respect to each individual training point. Second, noise calibrated to have a standard deviation proportional to this sensitivity is added to the average gradient. This results in a training algorithm known as differentially private SGD (DP-SGD). Unfortunately (as discussed in Section 1), DP-SGD does not perform well for models with large parameter counts, which motivates the improvements proposed in the next section.

3. IMPROVING DIFFERENTIALLY PRIVATE MODELS WITH ACTIVE LEARNING:

This paper introduces the following high-level process to improve the performance of an existing DP classifier: First, find a public insensitive dataset containing relevant samples. Second, carefully select a subset of public samples that can improve the performance of the classifier. Third, fine-tune the existing classifier on the selected samples with labels. We want to perform the selection and the fine-tuning in a way which does not compromise the privacy guarantees of the existing classifier.

The first step can be done using domain knowledge about the classifier, e.g., utilizing relevant public genomic datasets for a DP classifier of genomics data. We assume standard fine-tuning techniques for the last step. Therefore, the problem boils down to efficiently selecting samples from the public dataset while preserving privacy. We also assume a limit on the number of selected samples. This Cluster projected embeddings to get N cluster clusters D labeled ← ∅ for i = 1 to N cluster do Label N each data points from each cluster

D labeled = D labeled ∪ TakeCenterPoints(ClusterCenters) end for M * dp ← FineTune(M dp , D labeled )

limit is relevant when the samples are unlabeled, in which case it controls the cost of labeling (e.g., hiring human annotators to process the selected samples).

We introduce two active learning algorithms, DIVERSEPUBLIC and NEARPRIVATE , for sampleselection. These algorithms make different assumptions about access to the private training data but are otherwise drop-in replacements in the end-to-end process.

3.1. PROBLEM STATEMENT AND BASELINE METHODS:

We are given a differentially private model M dp trained and tested on private sensitive data D train and D test respectively. The privacy cost of training M dp on D train is ε dp (we omit δ for brevity in this paper, but it composes similarly). And we have an extra set of public insensitive unlabeled data D public which can be utilized to further improve M dp . Given the number of extra data N labeled that we are allowed to request labels for, and the total privacy budget ε limit of the improved model M * dp , we want to efficiently pick D labeled ⊂ D public where |D labeled | = N labeled , using which we can fine-tune M dp to M * dp of better performance on D test . The simplest baseline method we will use is to choose N labeled 'random' samples out of D public for fine-tuning. The other baseline that we use is Uncertain Sampling (Settles, 2009) , according to either the 'entropy' of the logits or the 'difference' between the two largest logits. These are widely used active learning methods and serve as strong baselines.

3.2. THE DIVERSEPUBLIC METHOD:

The baseline methods may select many redundant examples to label. To efficiently select diverse representative samples, we propose the DIVERSEPUBLIC method, adapted from clustering based active learning methods (Nguyen & Smeulders, 2004) . Given a DP model M dp of cost ε dp , we obtain the 'embeddings' (the activations before the logits) E public of all D public , and perform PCA on E public as is done in Raghu et al. (2017) . Then we select a number (more than N labeled ) of uncertain points (according to, e.g., the logit entropy of the private model) from D public , project their embeddings onto the top few principal components, and cluster those projections into groups. Finally, we pick a number of samples from each representative group up to N labeled in total and fine-tune the DP model with these D labeled . Though this procedure accesses M dp , it adds nothing to ε dp , since there is no private data referenced and the output of DP models can be post-processed at no additional privacy cost. It can be applied even to models for which we cannot access the original training data. 4Algorithm 1 presents more details of the DIVERSEPUBLIC method.

3.3. THE NEARPRIVATE METHOD:

The DIVERSEPUBLIC method works well under the assumption that D public has the same distribution as D train . However, this may not be a reasonable assumption in general (Oliver et al., 2018) . For instance, there may be a subset of D public about which our pre-trained model has high uncertainty but which cannot improve performance if sampled. This may be because that subset contains corrupted data or it may be due simply to distribution shift. In order to mitigate this issue, we propose to (Dwork et al., 2014b) on E train at a privacy cost of ε dpPCA . We select a number of uncertain points from both D train and D public . Then in the space of low dimensional DP-PCA projections, we assign each uncertain private example to exactly one uncertain public example according to Euclidean distance. This yields, for each uncertain public example, a count N support of 'nearby' uncertain private examples. Finally, we choose N labeled points with the largest values of N support + LaplaceNoise(1/ε support ) from the uncertain public data. This sampling procedure is differentially private with cost ε support due to the Laplace mechanism (Dwork et al., 2014a) . The total privacy cost of NEARPRIVATE is composed of the budgets expended to perform each of the three operations that depend on the private data: ε dp + ε dpPCA + ε support . More details in Algorithm 2.

4. EXPERIMENTS:

We conduct experiments on the MNIST (LeCun et al., 1998) and SVHN (Netzer et al., 2011) data sets. These may be seen as 'toy' data sets in the object recognition literature, but they are still challenging for DP object recognizers. In fact, at the time of this writing, there are no published examples of a differentially private SVHN classifier with both reasonable accuracy and non-trivial privacy guarantees. The baseline we establish in Section 4.2 is thus a substantial contribution by itself. For both datasets, we use the same model architecture as in the Tensorflow Privacy tutorials 5 . We obtain M dp by training on D train via DP-SGD (Abadi et al., 2016) . Unless otherwise specified, we always aggregate results over 5 runs with different random seeds and use error bars to represent the standard deviation. We use the implementation of DP-SGD made available through the TensorFlow Privacy library (McMahan et al., 2018) with δ = 10 -5 .

4.1. EXPERIMENTS ON MNIST:

We conduct our first set of experiments on the MNIST (LeCun et al., 1998) dataset. We use the Q-MNIST (Yadav & Bottou, 2019) dataset as our source of public data. In particular, we use the 50,000 examples from the Q-MNIST dataset that are reconstructions of the lost MNIST testing digits. We perform a hyperparameter optimization and find a baseline model with higher accuracy (97.3% at ε dp = 3.0, and 97.5% at ε dp = 3.2) than what is reported as the current state-of-the-art in the Tensorflow Privacy tutorial's README file (96.6% at ε dp = 3.0).

Figure 1a shows results of our DIVERSEPUBLIC method compared with baselines. Starting from a checkpoint of test accuracy 97.5% (ε dp = 3.2), our method can reach 98.8% accuracy with 7,000 extra labels. The DIVERSEPUBLIC method yields higher test accuracy than other active learning baselines in the low-query regime, and performs comparably in the high-query regime.

In Figure 1b , we compare NEARPRIVATE against DIVERSEPUBLIC . Since NEARPRIVATE adds extra privacy cost, we have to take special care when comparing it to DIVERSEPUBLIC . Therefore, we finetune from a starting checkpoint at test accuracy 97.0% with lower privacy cost (ε dp = 2.2) and make sure its total privacy cost (ε limit = 3.2) in the end is the same as the cost for the DIVERSEPUBLIC model. For this reason, NEARPRIVATE takes some number of labeled data points to 'catch up' to DIVERSEPUBLIC for the same DP cost -in this case 2,000. When N labeled is large enough, NEARPRIVATE outperforms all other methods. This shows that accessing the original training data in a privacy-aware way can substantially improve performance.

4.2. EXPERIMENTS ON SVHN:

We conduct another set of experiments on the SVHN (Netzer et al., 2011) data. We use the set of '531,131 additional, somewhat less difficult samples' as our source of public data. Since a baseline model trained with DP-SGD on the SVHN training set performs quite poorly, we have opted to first pre-train the model on rotated images of D public predicting only rotations as in Gidaris et al. (2018) .

Broadly speaking, the results presented in Figure 2 are similar to MNIST results, but three differences stand out. First, the improvement given by active learning over the baseline private model is larger. Second, the improvement given by DIVERSEPUBLIC over the basic active learning techniques is also larger. Third, NEARPRIVATE actually underperforms DIVERSEPUBLIC in this case. We hypothesize that the first and second results are due to there being more 'headroom' in SVHN accuracy than for MNIST, and that the third result stems from the reported lower difficulty of the extra SVHN data. In the next section, we examine this phenomenon further.

4.3. EXPERIMENTS WITH DATASET POLLUTION:

We were intrigued by the under-performance of NEARPRIVATE relative to DIVERSEPUBLIC on SVHN. We wondered whether it was due to the fact that SVHN and its extra data violate the assumption built into NEARPRIVATE -namely that we need to query the private data to 'throw out' unhelpful public data. Indeed, the SVHN website describes the extra set as 'somewhat less The setup here is analogous to the one in the MNIST experiment, but DIVERSEPUBLIC performs better in this case. Since we spent ε dpPCA + ε support = 1.0 on NEARPRIVATE , we fine-tune from 'checkpoint 0' with test accuracy 73.5% and ε dp = 5.0.

difficult' than the training data. To test this hypothesis, we designed a new experiment to check if NEARPRIVATE can actually select more helpful samples given a mixture of relevant data and irrelevant data as 'pollution'. We train the DP baseline with 30,000 of the SVHN training images, and treat a combination of another 40,000 SVHN training images and 10,000 CIFAR-10 ( Krizhevsky et al., 2009) training images as the extra public dataset. These CIFAR-10 examples act as the unhelpful public data that we would hope NEARPRIVATE could learn to discard. As shown in Figure 3 , all baselines perform worse than before with polluted public data. DIVERSEPUBLIC does somewhat better than random selection, but not much, achieving a peak performance improvement of around 1%.

On the other hand, the difference between NEARPRIVATE and DIVERSEPUBLIC is more substantial, at over 2% accuracy in some cases. This is especially interesting considering that DIVERSEPUBLIC actually performed better in the results of Section 4.2. Broadly speaking, the results support our claim that NEARPRIVATE helps more relative to DIVERSEPUBLIC when there is 'unhelpful' data in the public dataset. This is good to know, since having some unhelpful public data and some helpful public data seems like a more realistic problem setting than the one in which all public data is useful.

5. ABLATION ANALYSES:

In order to better understand how the performance of DIVERSEPUBLIC and NEARPRIVATE is affected by various hyper-parameters, we conduct several ablation studies.

5.1. HOW DO CLUSTERING HYPER-PARAMETERS AFFECT ACCURACY?:

For DIVERSEPUBLIC , there are two parameters that affect the number of extra data points labeled for fine-tuning: the number of clusters we form (N cluster ) and the number of points we label per cluster (N each ). We write N labeled = N cluster × N each . To study the relative effects of these, we conduct the experiment depicted in Figure 4a . In this experiment, we fine-tune the same DP MNIST model (test accuracy 97% at ε dp = 2.2) with varying values of N cluster and N each . We vary N cluster from 100 to 500, which is depicted on the x-axis. We vary N each from 5 to 20, with each value depicted as a different line. The general trend is one of diminishing returns on extra labeled data, as would be predicted by Hestness et al. (2017) . We do not notice a strong correspondence between final test accuracy at a fixed number of extra labels and the values of N cluster and N each . This is encouraging, as it suggests that practitioners can use our techniques without worrying too much about these hyper-parameters. Left: We fix the extra privacy cost ε dpPCA + ε support = 1.0, but vary the number of labeled public points for each line. We achieve large improvements from 1,000 to 4,000 labeled public points. With even larger N labeled , the improvements are not significant and it may be better to start fine-tuning from a DP checkpoint of lower privacy cost. Right: With ε dpPCA set to 0.3, we fix the total privacy cost ε limit = 5.0 and vary ε support . For N labeled = 1000, fine-tuning from the checkpoint at Epoch 60 is the best given a total privacy budget of 5.0.

5.2. HOW DOES THE STARTING CHECKPOINT AFFECT RESULTS?:

Recall that NEARPRIVATE accrues extra privacy cost by accessing the histogram of neighbor counts. This means that achieving a given accuracy under a constraint on the total privacy cost requires choosing how to allocate privacy between NEARPRIVATE and the initial DP-SGD procedure. Making this choice correctly requires a sense of how much benefit can be achieved from applying NEARPRIVATE to different starting checkpoints. Toward that end, we conduct (Figure 5 ) an ablation experiment on MNIST where we run NEARPRIVATE on many different checkpoints from the same training run. Figure 5a shows the test accuracies resulting from fine-tuning checkpoints at different epochs (represented on the x-axis) with fixed extra privacy cost of 1.0. Figure 5b shows the corresponding ε dp for each checkpoint. Figure 5c varies other parameters given a fixed total privacy budget of 5.0.

In Figure 5a , the black line with triangle markers shows the initial test accuracies of the checkpoints.

The other lines show results with different values of N labeled from 1,000 to 10,000 respectively. With N labeled = 1000, improvements are marginal for later checkpoints. In fact, the improvement from using N labeled = 1000 at checkpoint 80 is not enough to compensate for the additional 1.0 privacy cost spent by NEARPRIVATE , because you could have had the same increase in accuracy by training the original model for 20 more epochs, which costs less than that. On the other hand, the improvements from using N labeled = 4000 or higher are significant and cannot be mimicked by training for longer.

Given a total privacy budget ε limit , how should we decide among ε dp , ε dpPCA , and ε support ? Empirically, we observe that ε dpPCA can be set to a small value (e.g., 0.3) without substantially affecting the results. Allocating between ε dp and ε support is addressed in Figure 5c , which varies those parameters with ε limit fixed to 5.0. When the budget for gathering D labeled is low, say N labeled is around 1000, it is preferable to pick a later DP checkpoint, consuming a higher ε dp and lower ε support . On the other hand, when allowed to label more instances from the public data, we should use an earlier DP checkpoint (with a lower ε dp ) and choose better public samples with respect to the private data.

6. CONCLUSION:

In addition to creating new baselines for DP image classifiers by fine-tuning on public data, we introduce two algorithms -DIVERSEPUBLIC and NEARPRIVATE -to perform fine-tuning in a privacyaware way. We conduct experiments showing that these algorithms bring DP object recognition closer to practicality, improving on the aforementioned benchmarks. We hope that this work will encourage further research into techniques for making differential privacy more useful in practice, and we hope that the techniques we propose here will be helpful to existing practitioners.

:

For example, the best performing CIFAR-10 classifier fromZagoruyko & Komodakis (2016) has 32.5 million parameters, while the private baseline we describe in Section 4.1 has about

thousand.2 If there is labeled public data, the situation is even better, but this is likely to be rare and is not the setting we consider here.

URL blinded for anonymity.

e.g., models published to a repository like TensorFlow Hub: https://www.tensorflow.org/hub

Tutorials for TensorFlow Privacy are found at: https://github.com/tensorflow/privacy

Figures and Tables:

Figure 1: MNIST experiments. All plots show test-set-accuracy vs. the number of extra points labeled. Left: Comparison of our DIVERSEPUBLIC technique with the active learning baselines described above. All models are fine-tuned starting from the same baseline ('checkpoint 1': test accuracy 97.5% and ε dp = 3.2). Active learning improves the performance of the DP-model to as much as 98.8% in the best case with no increase in privacy cost. Right: Comparison of our NEARPRIVATE technique with the DIVERSEPUBLIC technique. Since we spent ε dpPCA +ε support = 1.0 on the NEARPRIVATE technique, we fine-tune the NEARPRIVATE model from 'checkpoint 0' with privacy cost ε dp = 2.2. Thus, both lines have the same privacy cost (ε limit = 3.2), regardless of the number of extra points used.

Figure 2: SVHN experiments. Left: Comparison of our DIVERSEPUBLIC technique with the active learning baselines. All models are fine-tuned starting from the same baseline ('checkpoint 1': test accuracy 75.0% and ε dp = 6.0). Active learning improves the performance of the DP-model to as much as 85% in the best case with no increase in privacy cost. Recall also that the 75.0% number is itself a baseline established in this paper. Right: Comparison of NEARPRIVATE with DIVERSEPUBLIC .The setup here is analogous to the one in the MNIST experiment, but DIVERSEPUBLIC performs better in this case. Since we spent ε dpPCA + ε support = 1.0 on NEARPRIVATE , we fine-tune from 'checkpoint 0' with test accuracy 73.5% and ε dp = 5.0.

Figure 3: SVHN experiments with dataset pollution. In this experiment, we train the DP baseline with 30,000 of the SVHN training images. The extra public dataset is a combination of 40,000 SVHN training images and 10,000 CIFAR-10 training images. Left: DIVERSEPUBLIC compared against other active learning baseline techniques as in Figure2a. In this case, the active learning techniques do not outperform random selection by very much. Uncertainty by itself is not a sufficient predictor of whether extra data will be helpful here, since the baseline model is also uncertain about the CIFAR-10 images. Right: NEARPRIVATE compared against DIVERSEPUBLIC , but started from different checkpoints ('checkpoint 0' with ε dp = 5.0, 'checkpoint 1' with ε dp = 6.0) to keep the privacy cost constant as in Figure2b. In this case, NEARPRIVATE substantially outperforms DIVERSEPUBLIC by selecting less of the CIFAR-10 images.

(a) DIVERSEPUBLIC with different Ncluster and Neach. (b) Visualization of chosen public data points.

Figure 4: DIVERSEPUBLIC analysis. Left: We apply DIVERSEPUBLIC to the same DP checkpoint (dashed horizontal line), varying the number of clusters (horizontal axis) and the number of chosen points from each cluster (lines with error bars). Right: For N cluster = 100, N each = 20, we visualize the most central example from each cluster. Since NEARPRIVATE does not have explicit clustering, we use DIVERSEPUBLIC for this visualization. Green borders mean that the initial checkpoint (dashed horizontal line) predicted correctly; while red bordered examples (originally predicted incorrectly) have dots on the left showing predictions and dots on the right showing true labels.

(a) Fixed extra privacy cost with εdpPCA = 0.5, εsupport = 0.5. (b) The initial training privacy cost εdp for different checkpoints. (c) Fixed total privacy cost with εlimit = 5.0, εdpPCA = 0.3.

Figure 5: NEARPRIVATE analysis. We apply NEARPRIVATE to DP checkpoints (horizontal axis) of different privacy cost ε dp . The black line with triangle represents those starting checkpoints. Middle: The initial training privacy cost ε dp of DP checkpoints at different epochs of a single training run.Left: We fix the extra privacy cost ε dpPCA + ε support = 1.0, but vary the number of labeled public points for each line. We achieve large improvements from 1,000 to 4,000 labeled public points. With even larger N labeled , the improvements are not significant and it may be better to start fine-tuning from a DP checkpoint of lower privacy cost. Right: With ε dpPCA set to 0.3, we fix the total privacy cost ε limit = 5.0 and vary ε support . For N labeled = 1000, fine-tuning from the checkpoint at Epoch 60 is the best given a total privacy budget of 5.0.

Algorithm 1 The DIVERSEPUBLIC AlgorithmInput: M dp with privacy cost ε dp , D public , N labeled , k > N labeled , p, N cluster , N each Output: M * dp fine-tuned on selected public data with the same privacy cost ε dp E public ← M dp (D public )[-1]

Table content: <table><tr><td/><td>Compute 'embeddings' of public data</td></tr><tr><td>P ← PCA(E public , p)</td><td>Perform PCA on embeddings and get first p PCs</td></tr><tr><td>S public ← P × SelectUncertain(D public , k)</td><td>Project k most 'uncertain' public data onto PCs</td></tr><tr><td>ClusterCenters ← K-Means(S public )</td><td/></tr></table>

Algorithm 2 The NEARPRIVATE Algorithm Input: M dp with privacy cost ε dp , D public , N labeled , k > N labeled , p, D train , ε dpPCA and ε support Output: M * dp fine-tuned on selected public data with privacy cost ε dp + ε dpPCA + ε support E train ← M dp (D train )[-1] Compute 'embeddings' of private data P ← DP-PCA(E train , ε dpPCA , p) Perform DP-PCA on embeddings and get first p PCs S train ← P × SelectUncertain(D train , k) Project k most 'uncertain' private data onto PCs S public ← P × SelectUncertain(D public , k) Project k most 'uncertain' public data onto PCs NeighborCounts ← ∅ for a ∈ S train do In PC-space, assign each private point to exactly one public point Increment(NeighborCounts, NearestNeighbor(a, S public )) end for for b ∈ S public do Compute support with Laplacian noise for each public data point N support (b) ← NeighborCounts(b) + LaplaceNoise(1/ε support ) end for D labeled ← TakeArgmaxPoints(N support ) Label N labeled data points of the highest N support M * dp ← FineTune(M dp , D labeled ) check query points against our private data and decline to label query points that are too far from any training points (in projected-embedding-space). But doing this check while preserving privacy guarantees is nontrivial, since it involves processing the private training data itself in addition to M dp . Given a DP model of cost ε dp , we obtain the embeddings E train of all D train , and perform differentially private PCA (DP-PCA)

Table content: <table/>

Peer review (iclr2020):
---SUMMARY--- This paper gives two methods for improving an existing differentially private model. Both methods assume a source of unlabeled data that requires no privacy guarantees. The two methods make different uses of this public data to augment the original private model. Both preserve (a degree of) privacy for the original model. The first method is DiversePublic. DiversePublic only post-processes the original private model M -- in particular, it does not touch the original data -- and so its privacy is immediate. At a high level, DiversePublic works by picking the *best* unlabelled data points for augmenting M. First, it applies M to the unlabeled data and then applies PCA to the result. Next, it selects highly *uncertain* points, projects them onto the top k principal components, and clusters the results. Finally, it selects samples from each cluster and uses them to augment M. This augmenting process, called FineTune, is left as a black box. The second method is NearPrivate. The given motivation for NearPrivate is that the public dataset may have a different distribution than the original private training data. To address this, NearPrivate uses DP-PCA (a pre-existing technique for differentially private PCA) on the private dataset and projects both the private and public data onto the top k principal components. Then it compares uncertain points in the public and private dataset projections and only augments M (again using the black box FineTune) using uncertain public examples with many nearby uncertain private samples. Intuitively, this should select for points that are not too different than the original private training data. The paper then evaluates these algorithms with experiments on MNIST and SVHN under reasonable privacy parameters (I think -- see below). These experiments compare the performance of DiversePublic and NearPrivate, where both train M using vanilla DP-SGD [Abadi et al. 2016]. DiversePublic and NearPrivate trade performances on MNIST and SVHN, but NearPrivate does (as might be expected) better when the public dataset is polluted. In summary, the paper suggests DiversePublic and NearPrivate as useful ways to augment a given differentially private model M using public data, and their experiments suggest this is reasonable. ---DECISION--- Reject. This is an empirical paper. It proposes some algorithms and justifies these algorithms through experiments. However, the paper makes a confusing omission: it does not mention PATE [1, 2]. [1] *Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data*. Papernot et al. 2017. [2] *Scalable Learning with PATE*. Papernot et al. 2018. I am more of a pure differential privacy researcher and am therefore less familiar with this line of mostly empirical work. But as far as I can tell PATE works in the same setting as that of this paper: there is a private dataset, a public unlabelled dataset, and the goal is to train a model with a differential privacy guarantee for the private dataset. As a result, PATE seems directly relevant. In particular, the claim that *there are no published examples of a differentially private SVHN classifier with both reasonable accuracy and non-trivial privacy guarantees* seems wrong, as the PATE papers include just that. Moreover, PATE appears at least competitive with the algorithms here. For example, the comparison given as Table 1 in [2] suggests that PATE can get 98-99% accuracy on MNIST for privacy parameters at least as low as those used here with only a few hundred new labelings. In contrast, if I understand the experiments here, they are using thousands of new labelled examples to get the same accuracy and privacy. Similarly, on SVHN the experiments here get to 84% accuracy with 10,000 new labelled examples, whereas the same Table 1 puts PATE at 90% with almost identical privacy parameters and thousands fewer new labelled examples. That comparison is my main concern. Perhaps I am missing a reason why PATE is not comparable. If it is, it should certainly appear in the experiments. If PATE is indeed comparable, then the main improvement contributed by this paper is the post-processing aspect: PATE is a way to train a private model from scratch, but these methods work on an existing model. Even then, the question of how exactly the existing model is modified (the FineTune method) is unclear. More broadly, there are several parts of this paper that could be much clearer. I am sympathetic to paper length limitations, but I am certain many or all of these issues can be addressed in 8 pages. 1. DiversePublic description: What does it mean to *obtain the *embeddings*...E_{public}*? How is k picked? 2. How does the private point-public point assignment in NearPrivate work? Is it just random? How many points do we pick? 3. In general, how is FineTune meant to work? I understand it is meant to be an abstract black box in the algorithm descriptions, but the experiment description doesn*t explain it either. How does FineTune work in the experiments? 4. What does *headroom* mean in Section 4.2? 5. The way privacy parameters are specified at various points in the experiment section makes things hard to read. A concise table summarizing privacy parameters, accuracy, and additional labels would help. But overall, I am most interested in the PATE comparison. ---EDIT AFTER AUTHOR RESPONSE--- I missed the data-dependent aspect of PATE*s privacy guarantee. That is a point in favor of this paper. As such, I*ve increased my score from reject to weak reject. I actually think a revised version of this paper could reasonably appear in ICLR or similar venues. However, I think the necessary revisions are too large/numerous to accept the paper in its current form, even with promises of revisions. Here are a few revisions that I think would make the paper a much better submission to future conferences: 1. Add discussion of PATE. The data-dependence of their privacy guarantees is well-taken. But it is not clear that data-dependent privacy guarantees are as *trivial* as the paper currently claims with *there are no published examples of a differentially private SVHN classifier with both reasonable accuracy and non-trivial privacy guarantees* suggests. (In fact, I find it confusing that one of the PATE authors agrees with this claim -- they think their own work is trivial?). I agree that data-independent privacy guarantees are preferable, but this should certainly be discussed. As the authors themselves note, the original PATE paper was well-received, so if this paper wants to claim its privacy guarantees are somehow meaningless or even just unsatisfactory, that claim needs to be defended. 2. Elaborate on the fine-tuning process. Perhaps *fine tuning* is indeed *super standard in the object recognition literature*, but as all three reviews here indicate, the presentation of fine tuning is unclear in this paper. The author responses also seem to simultaneously claim that fine tuning is both *super standard* and *[not] obvious*. 3. To make room for the text above, it*s not clear that the material about data pollution or even presenting both methods is necessary. I would prefer to see a thorough explanation of the active learning/post-processing paradigm through one algorithm. The authors seem to want to claim that active learning is a useful private approach because it gets data-independent privacy guarantees and performance similar to algorithms with data-dependent privacy guarantees. A paper that focuses on and thoroughly defends that claim actually sounds pretty good!