Differentially Private Image Classification by Learning Priors from Random Processes
Authors: Xinyu Tang
Ashwinee Panda
Vikash Sehwag
Prateek Mittal
Venue: NeurIPS 2023 spotlight (2023)
Keywords: Differential privacy
image classification
deep learning
TL;DR: We provide new SOTA methods for DP image classification when training from scratch by using image priors

Abstract: In privacy-preserving machine learning, differentially private stochastic gradient descent (DP-SGD) performs worse than SGD due to per-sample gradient clipping and noise addition.
A recent focus in private learning research is improving the performance of DP-SGD on private data by incorporating priors that are learned on real-world public data.
In this work, we explore how we can improve the privacy-utility tradeoff of DP-SGD by learning priors from images generated by random processes and transferring these priors to private data. 
We propose DP-RandP, a three-phase approach. 
We attain new state-of-the-art accuracy when training from scratch on CIFAR10, CIFAR100, MedMNIST and ImageNet for a range of privacy budgets $\\varepsilon \\in [1, 8]$. In particular, we improve the previous best reported accuracy on CIFAR10 from $60.6 \\%$ to $72.3 \\%$ for $\\varepsilon=1$.

Differentially Private Image Classification by Learning Priors from Random Processes
Xinyu Tang" Ashwinee Panda" Princeton University
Vikash Sehwag
Prateek Mittal
Abstract
In rrivacy-preserving machine learning, diffcrentially private stochastic gradient descent (DP-SGD) performs worse than SGD due to per-sample gradient clipping and noise addition. recent focus in private learning research is improving the performance of DP-SGD on private data by incorporating priors that are learned real-world public data. In this work, we explore how we can improve the privacy- utility tradeoff of DP-SGD by learning priors from images generated by random processes and transferring these priors to private data. We propose DP-RandP, three-phase approach. We attain new safe--of-the-art accuracy when training from scratch on CIFARIO, CIFAR 100, MedMNIST and ImageNet for range of privacy budgets [1,8]. particular, we impoove the previous best reported accuracy on CIFARIO 10 from 60.6% to 72.3% for = 1. Our code is available at httpss/ggiubb.com//ssiiieenggont
Phase-I: Learning image prior with represontation learning (no pritury cos))
85
Synthetic inuges mislore
80
Sabeeperoond Irclsing
Test Accuracy
73
Fenture colncterr
Phase-Il: Training bead clessifier on top of frozen festines (ansil pornary budget
Privads oisuset
Fend classifer
Phase-Iil: Training both feature extractor and classtfir (rensining prinary budget 6)
Our 2P-
Ourprivats srabing
Hold
Private ditasee
Slasss 2322
Dormann
Hoad clasalior
Figure Comparing our results CI- FARIO and previous state-of-the-art for (e, 10 10-50-5-D- 3-DP setup. Our full DP-RandP outperforms all previous works on this commonly benchmarked dataset and re- duces the privacy cost needed to achieve 80% accuracy from 6 to E Even our private linear probing from noise prior outperforms all previous work at crucially has diminishing returns, short- coming that we address with our proposed three- -hese DP training framework (DP. RandP).
Our proposed DP training pipeline (DP- RandP) has three distinct phases. In Phase I, we sample images from random processes and train feature ex- tractor with representation learning to embed image priors beneficial for visual tasks. Phase we spend small privacy budget to train a linear classifier on top of extracted features of private data. In Phase III, we update all parameters with our remaining privacy bud- get. We demonstrate that incorporating image priors Phase-I significantly improves DP training and adopting Phase II before training the whole network in Phase III further improve test accuracy in DP training.
Figure 1:
but
'Equal contribution
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Introduction
Machine learning models are susceptible to a range of attacks that exploit data leakage from trained models for objectives such as training data reconstruction and membership inference (58] 4. Differen- tial Privacy (DP) is the gold standard for quantifying privacy risks and providing provable guarantees against attacks [20] 22. DP implies that the outputs of an algorithm e.g., the final weights trained by stochastic gradient descent (SGD) do not change much (given by the privacy budget E) across two
neighboring datasets D and D' that differ in a single entry.
Definition (Diffferntiall Privacy) A randomized mechanism M with domain D and range R preserves (e, -)dfferrrlll privacy iff for any two neighboring datasets D, D' E 'D and for any subset S C R we have Pr[M(D) E S] et Pr[M(D') E S] +
Differentially Private Stochastic Gradient Descent (DP-SGD) 159 is the standard privacy- preserving training algorithm for training neural networks private data, with an update rule given by ((++1) (+)
(Eie (Ve(x,, w(()) + of) where the changes to SGD [
Ji
and addition of noise sampled from d-dimensional Gaussian distribution & (0, 1) with standard deviation DP-SGD introduces bias and variance into SGD and therefore degrades utility, creating a challenging privacy-utility tradeoff. For example, the state-of-the-ar accuracy in private training is only 60.6%
are the par-s gradient clipping clip.(V((a; w(t)) = (ax
on CIFAR-10 at E = [35) while Dosovitskiy et al. 19 obtains 99.5% accuracy non-privately.
Theoretical analysis of the DP-SGD update yields that noise addition is especially harmful to convergence at the start of training [24], and that pretraining on public data can greatly improve
convergence in this initial phase of optimization [46] by providing better initialization [25]. Previous
works have improved the privacy-utility tradcoff of DP-SGD by pre- pre-training on large publicly
available datasets, such ImageNet [16]. to learn visual priors (50 49
54 99. Other works assume
that a small subset of in-distribution data is publicly available (68 2 45
52
Interestingly, previous work has uncovered that synthetic data learned from random processes 44
10
22 38
can be used in representation learning 330 12 to learn highly useful visual priors 39
S
Because there is a large distribution shift between synthetic images and natural images, training on synthetic images does not incur any privacy cost. In this work, we leverage noise priors learned from
synthetic images to boost the performance of DP training. and make the following key contributions:
We find that while noise priors are only marginally helpful in non non-private training. we can unlock their full potential to improve private training with a carefully calibrated training framework.
We provide empirical evidence that priors learned from random processes become more critical the privacy budget decreases, because priors provide fast convergence at the start of training, and have a much larger impact on private training than non-private training.
We find that training a single linear layer (linear probing) on top of a pretrained feature extractor that has learned noise priors is more robust large amounts of noise addition than end-to-er training of the entire network. We demonstrate this insight by linear probing with a small privacy budget = 0. to 57.1% on CIFARIO, achieving nontrivial performance with lower privacy cost than previous work has considered.
We observe that while linear probing from noise prior has diminishing returns performance as the privacy budget increases, end-to-end training of the entire network continues to improve with large but critically struggles for small privacy budgets.
We harness our insights by proposing a privacy allocation strategy that combines the benefits of learning from priors, linear probing, and full training into our full method DP-RandP (Visualized in Fig. Our proposed approach pretrains feature extractor on synthetic data learn priors from random processes without paying privacy cost, then pays a small privacy cost for linear probing and makes the best use of our remaining privacy budget by updating all parameters to adapt our learned features to the private data.
We evaluate DP-RandP against previous work and unlock new SOTA performance across CI- FAR 10, CIFAR 100, MedMNIST and ImageNet across all evaluated privacy budgets & E [1.8). We provide snapshot of our results in Fig.

---

2 DP-RandP: Mitigating the effects of noise in DP-SGD with noise prior
Synthetic image generation without natural images. Recent progress in computer vision shows that pretraining models on synthetic images without natural images 6 39 can learn visual priors that are competitive to priors from natural images. The synthetic images can either be generated from texture and fractal-like noise 6 391, or structure priors extracted from an untrained StyleGAN [3).
Noise prior from synthetic images for private training. We consider pretraining on synthetic data, generated by random processes [ 6 (example images are in Fig. 3), as a noise prior in differentially private training. We use contrastive representation learning [12] II 30 64], which aims learn features invariant to common image transformations, to learn good visual features using synthetic images. At high level, rather than using a random 'cold' initialization for our downstream private dataset, we are using representation learning to obtain a 'warm initialization" that encodes priors learned from random processes. Across common natural vision tasks, noise priors also ensure that there is no privacy leakage from the pre-training data into this 'warm initialization'.
Thes
Took
theise
Dord (Nc
ow2
(a) StyleGAN [3 (b) Shaders 6
Uppier
Figure 3: We synthetic
Figure 4: Comparison of training from initialization (cold) and pre-
Figure 5: We zoom in on a
data generated from random pro- cesses in representation learn- ing to learn useful image priors. Due the high distribution shift from real images, we do not in- cur privacy costs when learning noise priors from these images.
random 0, 9.3 with learning rate= 0.4 and find that the improve- ment of the warm start over cold start is mostly due to the initial- ization using synthetic dataset.
trained encoder on synthetic dataset (warm) across different privacy budgets (the x-axis the corresponding a).
How much noise prior benefits private training? As an initial exploration, we obtain a warm initialization via representation learning on synthetic images and compare with a competitive DP-SGD baseline [15) that uses cold (random) initialization (Fig. 4. We find that while the warm initialization only improves performance by 2.7% when a = 0, i.e., non-private training, the warm initialization improves performance by 12.5% when = 9.3 (equivalent to 1). In Fig. 5 we zoom in on a single point of comparison between o = 0 (dashed) and a = 9.3 (solid) for warm and cold initializations. We find that over the course of training,
more noise is added, both the
= 9.3 warm and cold initializations not only converge at the same rate (given an appropriately small learning rate) but also diverge from the non- non-private runs at similar rates. Although warm and cold start obtain different results in the private setting, the difference is mostly due to the initialization and is therefore magnified at smaller privacy budgets.
We support this claim by drawing connection to previous works. Ganesh et al. 25 prove the existence of out-of-distributin public datasets that can achieve small test loss on target private datasets. Bu ct al. (8] prove that in the gradient flow setting for the NTK regime, that is, when we are taking very small steps 77 - O, noise does not impact convergence. Mchta et al. (50), Panda et al. 54 propose scaling the learning rate 7) inversely with the noise a. We combine our analysis with these previous works by noting that if we start training from fixed initialization and set the learning
rate near-zero for small privacy budgets [50] 54 we enter the regime X where the noise does not impact convergence, and achieving nontrivial performance for these small privacy budgets provides
the first empirical evidence for the theory 25 that only the initialization matters.
We gather our insights into design goal that will enable us to achieve nontrivial performance under strict privacy constraints. We want to encode learned prior into our initialization and then adapt this prior to private data. We now introduce DP-RandP a method that achieves this key design goal.
3

---

2.1 Three-phase differentially private training framework
We propose three-phase DP training framework DP-RandP that has two phases after pretraining on synthetic images, and significantly improves the performance of DP training (Fig. D We first learn noise priors by training a feature extractor on synthetic data (Phase-I). We then split our private training into 1) Learning the head classifier with frozen features (Phase-II) and 2) End-to-end training of the entire network to co-adapt the feature extractor and head classifier (Phase-IlI).
Our design is motivated by the strengths and weaknesses of linear probing and end-to-end training in the private setting. First, the lg norm of the Gaussian noise added gradients in each DP-SGD step scales with the number of parameters in the model. Because the head classifier typically has far fewer parameters than the feature extractor, updating this linear layer reduces the amount of added noise. Second, there is large distribution shift between synthetic data from random processes and natural images found in private datasets. Linear probing merely inherits the frozen pre- pre-trained features, but end-to-end training of all layers can improve the pre-trained features by adapting them to the private dataset. Because adapting to our private dataset may require many end-to-end training steps, we improve the convergence by first learning task-specific linear classifier that can be trained quickly, and then training the entire network end-to-end. DP-RandP satisfies our previously stated design goal by first obtaining good initialization with pretraining, then transferring the prior encoded in this initialization to the private dataset with fast-converging linear probing, and finally end-to-end training to adapt to the private dataset as much as possible for given privacy budget.
We further consider the design choice of how to allocate the privacy budget for the two private training phases (E1 and E2). Recall that our model is pre-trainn pre-trained on synthetic data in Phase I, thus it doesn't 't incur any privacy cost. In Phase II, we use DP-SGD with privacy budget E to train the linear classifier. We observe diminishing returns in accuracy with increasing E1, so we spend a smaller privacy budget on linear classifier training and allocate the rest to training the full network in Phase III. Allocating higher budget to full training does not have diminishing returns; consider that &2 = co. we recover non- non-private accuracy. We now rigorously evaluate the performance of DP-RandP.
3 Evaluation
To outline the evaluation section we first overview our experimental setup and then evaluate the performance of DP-RandP CIFARIO/CIFAR 00/DeraaMNIST in Sec. 3.2 We find that our method outperforms all previous works across multiple datasets, architectures, and privacy budgets. In Sec. B.3 we find that principled allocation of privacy budget E2 between linear probing and end-to-end training is robust to different choices of é and E2. We also find that the private linear probing is strong computationally efficient baseline. Specifically, we provide a new SOTA on ImageNet with private linear probing. We next provide a quantitive comparison of DP-RandP to DP with public data. Finally we discuss the computational costs of our method and find that DP-RandP can provide computational savings over previous methods.
3.1 Experimental setup
Learning priors from images generated by random processes with representation learning. In Phase we sample images from StyleGAN-oriented [5], and train a feature extractor on these synthetic datasets with representation learning 112 64] with the loss function proposed in Wang and Isola [64). Although our method can accommodate any kind of synthetic data and representation learning method, we focus our evaluation these datasets and methods. We consider other kinds of synthetic data and other representation learning in Appendix A
Datasets and models. We evaluate DP-RandP on CIFAR IO/CIFARIOO [41], DermaMNIST in MedMNIST [65 66] and private linear probing version of DP- on ImageNet 16] For CIFAR10/CIFARIOO we use WRN-16-4 following De et al. [15]. For MedMNIST, we use R following Hölzl al. [34]. We choose WRN-16-4 and ResNet-9 because these architectures for
the corresponding datasets achieve the most compelling results in previous works [15] 34]. For
ImageNet, we use ViT-base [9] feature extractor pretrained on Shaders-21k (6) provided by Yu et al. [71] and train a linear classifier. We provide the results on CIFAR10, CIFAR 100, DernaMNIST
in Sec. 3.2 and results on ImageNet in Sec. 3.3 We also report results of WRN-40-4 on CIFARIO 10 in
Appendix B

---

Implementation details. To ensure fair comparison with previous works [13] 57 35], standard DP-SGD [ and make use of multiple data augmentations and exponential moving average (EMA) proposed by De et al.
we use
[5] that are now standard techniques used in DP-SGD work [77 35 We allocate small privacy budget $1 to Phase II and remaining privacy budget E2 to Phase III according to the strategy detailed in Sec. 3.3 We report our results across different privacy costs and use = 1008 for CIFAR 1O/CIFAR DeNT by following previous works [15] 34) When we report results, report the standard deviation and accuracy averaged across 5 independent runs with different random seeds. We report implementation details for CIFARIOCCIAAR I0CEFAAARRIO/DDMMMNNSS in Appendix and for ImageNet in Appendix G
3.2 Evaluation of DP-RandP
We report the results of DP-RandP in Tab. DermaMNIST datasets, respectively.
3 for CIFARIO, CIFARI00 and [ Tab. and Tab.
DP-RandP outperforms all previous works across all privacy budgets. In Tab. we find that
DP-RandP obtains higher performance than previous works [T. (5| 61
40
18
69 on CIFARIO
across the standard evaluated privacy budgets E [1.
We first compare DP-RandP to De et al. 15 as our CIFARIO model, optimizer and hyperparameters follow De et al. [5]: the only difference is the use of Phase I and Phase II in DP-RandP to learn prior from synthetic data and allocate a small privacy budget to linear probing. Crucially DP-RandP outperforms De et al. [[3] by more than 15% for the important privacy budget = =.
Tramèr and Boneh 61
ScatterNet 553 to encode invariant image priors and Hölzl al. 35 use equivariant CNNs [4] to learn transform invariant features. DP-RandP shares the same intuition of leveraging invariant image priors these works [6 35). Instead of leveraging model architecture design for invariant features, we achieve this intuition by learning priors from images generated from random processes and design our three-phase framework optimize use of this prior. Although DP-RandP shares this intuition of leveraging invariant image priors, DP-RandP achieves 12% improvement over previous works 6 35 who both achieve 60% ε 1. Our improvement comes both from leveraging the priors from synthetic images and our design of three learning phases that makes the best use of priors. We provide a detailed comparison to Tramèr and Boneh [6]] in Sec. 4 where we explain why and how the feature prior we learn from synthetic data provides better results than the feature prior provided by their architectures.
Table 1: Test accuracy (%) of DP-RandP and comparison to previous work on CIFARIO. Not shown in this table are Klause al. 40. Dörmann et al. [18, Papernot et al. (55]. Yu et al. 69 because they achieve 71.5% at that are not on the pareto frontier of previous work.
7.5 70.1% at e 7.42, 66. 26.2% at E = 7.53 and 63.4% at e = 8 respectively.
Method
==1
== 2
=
8 = 4
=
8 8
6 = 00
Tramer and Boneh 61]
60.3
67.2
69.3
73.8"
De et al. [T5]
56.8
64.9
69.2
71.9
77.0
79.5
88.9
Hölzi et al. 35
60.59
71.86
75.96
78.27
80.26
81.62
Da-RandP
72.32 22
77.
77.25..07
79.99o.21
81.88o.27
84.01
85.26o.11
91.69
CIFAR100 results. Previous work has not provided results on training from scratch for CIFAR100. perhaps because well for = = 3. We believe this to be a competitive baseline, but DP-RandP outperforms it by more than 10% across E [3, 8). Although CIFAR 10 and CIFARIO0 100 are both benchmark computer vision datasets, we find CIFAR to be a much more challenging task for private learning. One possible
find in Tab. the DP-SGD baseline following De et al. [15] does not perform
"W run experiments for & = ac with per-sample gradient clipping but without noise, because we interested the ability of DP-RandP to mitigate variance. We note that there accuracy degradation from the non non-private baseline even at e = oo due to the bias introduced by per pample gradient clipping [37]. When we report = co results from previous work we mark them with * when previous work does not report whether the non-private baseline uses clipping.
5

---

explanation for this is that private classifiers struggle to distinguish between many classes because the added noise is more likely to shift the decision boundary. We use the same hyperparameters for CIFARIO 10 and CIFAR100, that are likely suboptimal for CIFAR 100, and the performance gap between E = and = oo may be mitigated if we train on CIFARTON for longer than we do on CIFAR10. We encourage the use of CIFAR 100 as a standard benchmark for private learning in the future because we find limited room for improvement on CIFARIO IO. particular, the private and non-private gap between
and oo for CIFARIO is only R 6% for DP-RandP.
Table 2: Test accuracy (%) of DP-RandP and comparison to DP-SGD baseline on CIFARIO0.
Method
==
E = 8
6 00
DP-SGD
30.73
34.45
39.66
44.22
66.68
Da-RandP
43.33o.15
46.40g.31
51.530,13
55.02p.21
71.68
DermaMNIST results. We have shown that DP-RandP outperforms previous work on the standard CV benchmarks of CIFARIO and CIFAR 100, and now consider the privacy sensitive medical dataset DermaMNIST. Although CIFAR a standard CV benchmark, there is limited previous work that evaluates on privacy sensitive data in CV such as medical images. In Tab. [3 we find that DP-RandP achieve improvements from 8 to 7.42 by up to 2. 2.78% over the DP-SGD baseline that we evaluate. Also, DP-RandP can achieve similar accuracy at g = 4 as the result of DP-SGD at E = 7.42, which reduces the privacy cost from e = 7.42 to E = 4. We also include the results of Hölzl et al. B4], that use equivariant neural networks [14]. OP-RandP is a uniform framework applicable to any model architectures. We leave the systematic investigation of more model architectures, such as equivariant neural network [14 35 in DP-RandP, for future work.
Table 3: We follow previous work 34 and report the validation accuracy (%) of DP-RandP on DeTmaMNIST. We also report the test accuracy in Appendix D
1
C
4 E
7.42
baseline in Hölzl et al. (34]
72.41
78.48*
best in Hölzl et al. 34
74.17
77.84"
DP-SGD we evaluated
69.00p.37
71.78p.87
74.08p.41
77.27
Da-RandP
71.78p.40
74.82p.55
75.919.29
79.26
We find that the prior we learn from synthetic data is applicable to standard benchmarks and medical images. One direction for future work is to validate the robustness of this prior across more datasets.
3.3 Allocating privacy budget in DP-RandP
72
In this subsection we analyze the privacy budgets of Phase (E1, linear probing) and Phase III (62, full training) and how to allocate the overall privacy budget among Phase Il and Phase III. Specifically, we will present new STOA on ImageNet with additional designs on linear probing.
66
- All budger: alliited to Phise Il
- All budge: alliited to Phise Tramer and Binch, 2022
64
De at. 2022 Hold at 2023
6
In Fig. 6 we observe that DP-RandP is robust to the key algorithmic design choice of how much privacy budget to allocate to Phase and Phase III (Please check Apendix E for more details of Fig. 6 and more results on different é).
50
0.0
0.2
0.4
0.6
0.8
1.0
D/C Tractton Total Frivacy Busipt Alotbed Phaae
Figure 6: The fraction of total privacy budget allotted to Phase II for & 1. The performance is stable across wide range of value 10 0.4). However, skip- ping either Phase-Il or Phase-Ill leads to suboptimal test accuracy in DP training.
In particular, we find that even the worst choice of e / provides results comparable to the previous SOTA on CI- FARIO at = = F We first investigate the behavior at each extrema and conclude general allocation strategy that provides good performance across CIFARIO, CIFARIOO and DermaMNIST.
è Due to computational constraints we did not calculate error bars for Fig. resulting in some nonconvexity.

---

Allocating the entire privacy budget to Phase II is competitive for small privacy budgets but provides diminishing returns. This corresponds to the right extreme FE 1) and is equivalent to doing linear probing on top of extracted features. We follow the training recipe in Panda et al. [54] and report the result of private linear probing in Tab. I for CIFARIO and Tab.
for ImageNet [6] respectively (d 7.8 x 10-7 is by 8 = 1/1 D for ImageNet). We provide the detailed experimental set-up for CIFARIO in Appendix [ and ImageNet in Appendix
CIFARIO results. We present the result of private linear probing on CIFARIO by including more conservative privacy constraints like ε 0.03. Our private linear probing can achieve 57.10% at E 0.1, that is comparable to the result of & 1 in De et al. [I5] that fully trains a WRN-16 16-4. Notably, previous work [1]] also trains a neural network on top of the handcrafted features using an untrained ScatterNet [53] DP-RandP is slightly better than their result at E = 3. We can see that our non-private baseline (74.05%) is slightly better than the non-private baseline (73.8%) in Tramèr and Boneh [6]] that shows that our feature extractor is better than the ScatterNet and therefore improves the performance under DP-SGD. We include a detailed comparison to Tramèr and Boneh [6]] in Sec. 4 However, this private linear probing variant of DP-RandP can only achieve 74.05% with no noise added, that is lower than the result at E 4 in De et al. [15, that shows that allocating all privacy cost to Phase II is a sub-optimal design choice.
Table Test accuracy (%) of our private linear probing and comparison to previous SOTA for private learning on top of extracted features 61 on CIFARIO. Note that, our result is training a linear layer on top of the extracted features. The result of previous SOTA is by training CNN on top of extracted features. Tramèr and Boneh [61] also report the
Table 5: Test accuracy (%) of our private linear probing with addi- tional designs on ImageNet.
private linear probing result 07.0% at E
3.
De et al. 15
32.4
0.03
0.1
0.2
0.5
1
2
3
Sander et al. [57]
39.2
SOTA
60.3
67.2
69.3
Ours (ViT) (Std.)
26.54
39.39
0.11
0.03
Ours 40.64 57.10
60.89
65.
10
67.78
69.92
71.08
(Std.) 2.59 0.42 0.26
0.21
0.17
0.09
0.14
ImageNet results. We achieve new SOTA results on ImageNNet. We achieve 39.39% accuracy at E 8 and the previous SOTA [57] is 39.2% at 8. We use Vit-base [19] model pretrained on Shaders-21k k (the model checkpoint is provided by Yu et al. (71). If we directly do Phase II with extracted features, we can achieve around 33% accuracy at E = 8, that is comparable to De et al. 11]. The direct linear probing result shows that linear probing (Phase II only after pretrained on synthetic data) is not enough for difficult tasks like ImageNet, and therefore updating full parameters in Phase III is necessary. While training the full ViT model large datasets like ImageNet needs much computation resources (for example, Sander ct al. (57) used 32 A100 GPUs), we could leverage the core concept of DP- DP-RandP and adapt it to computationally efficient version, i.c., we train linear layer with additional modifications. We then obtain a new SOTA result of 39.39% on the ImageNet-Il 1k validation dataset. We now briefly explain the two main modifications of -nnRa to achieve 39.3 39.39%. We provide a detailed explanation in Appendix G We emphasize that these modifications more for computational efficiency; if we had enough compute do full fine-tuning of the ViT on ImageNet with sufficiently large batch size, our original DP-RandP would still work.
Our first modification is to approximate full fine-tuning by linear probing on larger feature represen- tations that we create by aggregating intermediate representations from the network. This is because each block of vision transformers learns different representation. However, linear probing only takes the representation from the penultimate layer as input and therefore the final representation may not be sufficient to learn the task. The representation of the input image after each block in the ViT has both a temporal and feature dimension, so we pool over the temporal dimension to gather feature map of size (4, feature size). We concatenate the feature map of different blocks into one-dimensional vector. By doing linear probing on these much larger features, we can also take advantage of intermediate representations.
Our second modification is to approximate the work of a LayerNorm or other normalization layer that we would update during full fine-tuning of the entire ViT, by manually normalizing the features.
7

---

allocate the entire To do this we first normalize each feature vector to a fixed norm. We next privately estimate the mean over the entire ImageNet feature vector dataset, using the Gaussian mechanism with a small privacy cost, and subtract the private mean from all feature vectors. This is equivalent to doing non-private centering and then adding the same Gaussian noise to the entire dataset. This procedure can be thought of as a one-time approximation to the normalization layer, which is known to speed up training by centering the data.
Within the two modifications of direct linear probing, we improve upon previous SOTA 57 and achieve 39.3 39.39% at E 8. This method is computationally efficient and one run can be done on a single A100 GPU in a few hours. We also provide the result for a stronger privacy guarantee, i.e., 26.54% E
Allocating the entire privacy budget to Phase III struggles for small privacy budgets. We report DP-RandP without Phase II on CIFARIO in Tab. 6 (equals to E = 0 in Fig. 6 The result in Tab. 6 is slightly worse than DP-RandP in Tab. and the utility gap between Tab. 6 and Tab. decreases as E increases, that justifies the importance Phase II in DP-RandP. Moreover, the result
of DP-RandP without Phase II is significantly better than previous SOTA [5 35 61], that shows that the feature extractor pretrained on images from random process can capture the image prior.
Table 6: Fully privately training a WRN-16-4 with warm initialization. Test accuracy CIFARIO.
= 2
4
6
==b 8
Accuracy(%) (%)
69. 03.03 75.31o.28
78, 78.44..19
80.56p.12
82.90o.10
84.45%.09
91.69
general privacy budget allocation strategy. We have observed that allocating the entire privacy budget to linear probing or full training is suboptimal. We now propose a simple yet effective general strategy to allocate the privacy budget. For small (E < 1), we set 5 privacy budget to Phase II. This is because allocating the entire privacy budget to linear probing competitive for small privacy budgets. As E increases, we decrease f1/e. This is because as increases, the noise multiplier will decrease. Therefore, it is easier to train the linear probing layer as e increases, and the percentage of total steps allocated to Phase II can reduced to train a good linear probing layer and we can use the remaining steps for Phase III. Because the closed-form computation of e with numerical privacy loss distribution accounting by Gopi et al. (27 is challenging, we implement this strategy by using fixed number of steps for linear probing in CIFARIO, CIFAR100, DermaMNIST, and increasing the number of steps in Phase III as e increases. We present the privacy allocation /E CIFARIO in Appendix E for results in Tab. I which validates this strategy.
3.4 In comparison to DP with public data
major direction in improving the privacy utility trade-off for DP-SGD is by incorporating priors
that are learned real-world public data [68 52 50
49
54
[66
These priors either from
small in-distribution data [68
45
52
or large scale public data 50
54 56
Table 7: Comparison of DP-RandB with methods using public data. Test accuracy (%) on CIFAR1O.
Method
Model
Public Data
1
2
E 6
8
Nasr et al. 52
WRN-16-4
4% ID
72.10
75.10
77.9
80.0
Mehta et 500
ViT-B/16
ImageNet
95.10
95.10
95.10
95.20
Mchta et al. 49
ViT-G/16
JFT
98.80
98.80
98.83
98.84
Panda et al. [54]
beitv2
ImageNet
99.00
DP-RandP
WRN-16-4
Synthetic
72.32
77.25
81.88
84.01
85.26
We present a quantitative comparison of DP-RandP and DP with public data works 54 50
49
52
in
Tab. 7 Our Ra-RandP is comparable with Nasr et [52], which in fact utilizes a limited amount of in-distribution data as public data for pre-training. This indicates that the prior learned from images
8

---

generated from random processes can help much as the prior learned from limited in in-disributtion public data. Compared to works [50 between our DP-RandP and these works. Note that we consider different threat model to this line of work where we do not have access to public data [62 and must instead make the best possible use of images drawn from random processes. Closing the gap between leveraging synthetic data and leveraging large-scale real public data is an interesting direction for future work.
49 54 with access large public data, there is still a gap
3.5 Computational cost
DP-RandP consists of three phases. Some phases can be done in a single-run per dataset. For example, for feature extractor in Phase we only need train a single feature extractor once for each evaluated dataset as the synthetic dataset and model architectures are the same. Also, for the private linear probing (LP). We report the computation cost in Tab. for single-run per dataset. For Phase II and Phase III, we need to go through these processes at each run of our training process. Also, after we get the extracted features for LP,, we need to train linear layer each time we run the experiments. We report the computation cost in Tab. lab.99oo for these procedures.
De al. [15] also needs fully train a WRN-16-4 and the major additional computation cost for DP-RandP is Phase I compared to De et al. [15] However, the training in Phase I is done once for CIFAR1O. Moreover, we use the same feature extractor from Phase for CIFARIO and CIFAR100.
Table 8: One-time per dataset computational cost on CIFARIO. These procedures only need to be done once for each evaluated dataset. Phase I can be shared for CIFARIO /CIFAR100. such as De et al. (15].
Table 9: Computational cost CIFAR10 for hyper- parameters given in Appendix These procedures comparable the standard training procedure
Phase
feature extraction in LP
linear probing in LP
Phase II
Phase III
Time
16 h
min
Time
1 min
12min
5.5 h
For the private linear probing experiment, each run of training a linear layer can be finished in 1 minute for CIFARIO and 320 minutes for ImageNet while fully privately training model costs much more time. A single run to privately train a WRN-16-4 for CIFARIO 10 takes around 5.5 hours for 875 steps with 1 A100 GPU in our evaluation. Also, as reported in previous work [57). single run for ImageNet experiments needs to take four days using 32 A100 GPUs.
4 Discussion and related work
In this section we first provide detailed comparison previous work [61] that also uses priors to improve DP-SGD image classification. We then give an overview of the broader body of work on improving the privacy utility tradeoffs in DP-SGD. Finally we discuss the previous work [43 that also uses the two-stage training with domain-specific data for a different reasoning.
Discussion DP-RandP and Tramer and Boneh 61]. Tramèr and Boneh 661 find that training a neural network (linear layer or CNN) on top of 'handcrafted' ScatterNet [33] features outperforms private 'deep' learning. While this method performs well for smaller values of e, the non-private accuracy is limited because the features cannot be adapted to the private data. There are two key differences between DP-RandP and Tramèr and Boneh 61). In Phase I we representation learning to train the feature extractor on images sampled from random processes, to learn the prior that extract features. Our feature extraction process is therefore much more general, and while we explore the use of different synthetic data, model architectures, and representation learning methods, there are many more methods in each of these categories that we have not explored. The second difference is between the training on top of extracted features, that is used by Tramèr and Boneh [6]] and the private linear probing that we consider as baseline in Fig. [ and the combination of linear probing and full training that we use in RanRa. Comparing the improvements between DP-RandR and Tramèr and Boneh 61 confirms that our empirical improvements are mostly due to the advantage of DP-RandP over DP linear probing. In particular, for ε 3, exchanging the handcrafted features of Tramer and Boneh [6] for the pretrained feature extractor we use only improves performance by modest 2%. However, our full Randd cxhibits more than 10%
transformationsiivaarte

---

improvement. Our innovation over Tramèr and Boneh [6] is therefore twofold: we introduce the potential of pretraining on synthetic data for the DP community, and also provide guidance on how to better transfer features learned from synthetic data to private training.
DP with public data. A major direction in improving the privacy utility tradeoff in DP-SGD for image classification is the principled use of public data. Several works [68] 2 45 52 make use of public data under a different threat model by treating small fraction of the private training dataset as public. There is also another line of work that leverages a large real-world public dataset to pretrain [6].
models [50 49
Besides directly training image classification by DP-SGD, another direction is oP-trrined generative models. The generated images can be used for classification tasks without additional privacy costs. Recent work (26) show that DP diffusion models can achieve high-quality images when pretrained on large public data like ImageNet and achieve 88.8% classification accuracy for CIFAR1O at 10.
Another line of work has shown the success of DP-SGD fine-tuning of pretrained large language models (LLMs) [28 47 70 LLM pretraining can be framed as way to learn structural priors from unstructured data 7
DP-SGD fraining from scratch. In addition to the directly related works discussed above. the baselines for training from scratch that we compare to in this work are De al. [15] and Sander et al.
[57] De et al. [15] make use of multiple techniques that are now a mainstay of DP-SGD training from scratch such as multiple data augmentations [32 that we also use. Sander et [57 propose method for estimating the best hyperparameters for DP training at scale using smaller-scale runs. propose the use of inductive bias via architectural prior. They use DP-SGI) to train the equivariant CNN architecture [14] and achieve 81.6% at = 8 on 10. We note that the design space of novel architectures that are especially compatible with DP is rich and mostly unexplored, and our approach is compatible with any advancements in this domain. In particular, using the architecture of Hölzl et al. [4 35) in DP-RandP could potentially enjoy the improvements by combining the feature priors. We leave this exploration future work.
Recent works 34 35
Two-stage training with domain-specific data. The two-stage training of first training the classifier head and then tuning all hyperparameters has also shown to be effective for out-of of-distribution (OOD) tasks 43 Their reasoning is that if the full parameters are directly updated using the in-distribution training data, this may lead to the loss of some general features learned in the pretrained stage and result in utility drop on OOD data. Our task of DP image classification is different from the OOD task (43) but still shares some common intuition. After Phase I, our feature extractor has learned some useful priors while the classifier head has a random initialization. If we directly update the full network, too much noise is added to the full network, which may distort features learned in Phase I and lead to suboptimal performance. We also conduct experiments and find that, without noise prior (therefore model is randomly initialized), the two-stage training pipeline would not significantly improve the performance compared to directly training the full parameters (See Appendix H)).
5 Conclusion
We leverage images generated from random processes and propose a three-phase training DP-RandP to optimize the use of noise prior. The evaluation across multiple datasets including the benchmark datasets CIFARIO and ImageNet shows that DP-RandP can improve the performance of DP-SGD. For example, Randa improves the previous best reported accuracy on CIFARIO from 60.6% to 72.3% a Da-RandP is general framework for different datasets, models, and representation learning methods. Future improvements of designs in each of these categories would potentially improve the performance of DP- DP-RandP makes use of priors from synthetic images. would be interesting to study whether Randa would improve the priors for DP with public data. Also, investigating the priors beyond image domains, e.g., language and speech tasks, for differentially private training would also be of great interest.
Acknowledgements
We are grateful to anonymous reviewers at NeurIPS for valuable feedback. This work was supported in part by the National Science Foundation under grant CNS-2131938, the ARL's Army Artificial Intelligence Innovation Institute (A212), Schmidt DataX award, and Princeton E-ffiliates Award.
10

---

References
[1] Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, llya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security, pages 308-318, 2016.
[2] Ehsan Amid, Arun Ganesh, Rajiv Mathews, Swaroop Ramaswamy, Shuang Song. Thomas Steinke, Thomas Steinke, Vinith M Suriyakumar, Om Thakkar, and Abhradeep Thakurta. Public data-assisted mirror descent for private model training. In Proceedings of the 39th International Conference on Machine Learning, pages 517-535, PMLR, 2022.
[3] Borja Balle and Yu-Xiang Wang. Improving the gaussian mechanism for differential pri- vacy: Analytical calibration and optimal denoising. In Proceedings of the 35th International Conference on Machine Learning, pages 394-403, PMLR, 2018.
[4] Borja Balle, Giovanni Cherubin, and Jamie Hayes. Reconstructing training data with informed adversaries. In 2022 IEEE Symposium on Security and Privacy (SP), pages 1138-1156, 2022.
[5] Manel Baradad, Jonas Wulff, Tongzhou Wang, Phillip Isola, and Antonio Torralba. Learning to see by looking at noise. In Advances in Neural Information Processing Systems, pages 2556-2569, 2021.
[6] Manel Baradad, Richard Chen, Jonas Wulff, Tongzhou Wang, Rogerio Feris, Antonio Torralba, and Phillip Isola. Procedural image programs for representation learning. In Advances in Neural Information Processing Systems, pages 6450-6462, 2022.
[7] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In Advances in Neural Information Processing Systems, pages 1877-1901, 2020.
[8] Zhiqi Bu, Hua Wang. Zongyu Dai, and Qi Long. On the convergence and calibration of deep learning with differential privacy. Transactions on Machine Learning Research, 2023. ISSN 2835-8856.
[9] Zhiqi Bu, Yu-Xiang Wang, Sheng Zha, and George Karypis. Differentially private optimization on large model at small cost. In Proceedings of the 40th International Conference on Machine Learning, pages 3192-3218. PMLR, 2023.
[10] Geoffrey Burton and Ian R Moorhead. Color and spatial structure in natural scenes. Applied optics, 26(1):157-170, 1987.
[11] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In Proceedings of the 37th International Conference on Machine Learning, pages 1597-1607. PMLR, 2020.
[12] Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He. Improved baselines with momentum contrastive learning, arXiv preprint 20t.:2020004297. 2020.
[13] Christopher A. Choquette- Choo, Krishnamurthy Dvijotham, Krishna Pillutla, Arun Ganesh, Thomas Steinke, and Abhradeep Thakurta. Correlated noise provably beats independent noise for differentially private learning. arXiv preprint arXiv:23 2023.
[14] Taco Cohen, Maurice Weiler, Berkay Kicanaoglu, and Max Welling. Gauge equivariant convolutional networks and the icosahedral CNN. In Proceedings of the 36th International Conference on Machine Learning, pages 1321-1330. PMLR, 2019.
[15] Soham De, Leonard Berrada, Jamie Hayes, Samuel L Smith, and Borja Balle. Unlock- ing high-accuracy differentially private image classification through scale, arXiv preprint XXX1:2204.136533 2022.
11

---

[16] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248 248-255, 2009.
[17] Jinshuo Dong, Aaron Roth, and Weijie Su. Gaussian differential privacy, arXiv preprint arXiv:1955.02.0 1905.0233, 2019.
[18] Friedrich Dörmann, Osvald Frisk, Lars Nørvang Andersen, and Christian Fischer Pedersen. Not all noise is accounted equally: How differentially private learning benefits from large sampling rates. In 2021 IEEE 31st International Workshop on Machine Learning for Signal Processing (MISP), pages 1 1-6, 2021.
[19] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations, 2021.
[20] Cynthia Dwork. Differential privacy: A survey of results. In International Conference on Theooy and Applications of Models of Computation, pages 1-19, 2008
[21] Cynthia Dwork and Aaron Roth. The algorithmic foundations of differential privacy. Founda- fions and Trends@ in Theoretical Computer Science, (03-4)2111-4777 2014.
[22] Dumitru Erhan, Yoshua Bengio, Aaron Courville, and Pascal Vincent. Visualizing higher-layer features of deep network. University of Montreal, 1341(3):1, 2009.
[23] Utku Evei, Vincent Dumoulin, Hugo Larochelle, and Michael Mozer. Head2Toe: Utilizing intermediate representations for better transfer learning. In Proceedings of the 39th International Conference on Machine Learning, pages
PMLR, 2022.
[24] Huang Fang, Xiaoyun Li, Chenglin Fan, and Ping Li. Improved convergence of differential private SGD with gradient clipping. The Eleventh International Conference on Learning Representations, 2023.
[25] Arun Ganesh, Mahdi Haghifam, Milad Nasr, Sewoong Oh, Thomas Steinke, Om Thakkar, Abhradeep Guha Thakurta, and Lun Wang. Why is public pretraining necessary for private model training? In Proceedings of the 40th International Conference on Machine Learning. pages 10611-10627. PMLR, 2023.
[26] Sahra Ghalebikesabi, Leonard Berrada, Sven Gowal, Ira Ktena, Robert Stanforth, Jamie Hayes, Soham De. Samuel L. Smith, Olivia Wiles, and Borja Balle. Differentially private diffusion models generate useful synthetic images. arXiv preprint arXiv:2302. 13861, 2023.
[27] Sivakanth Gopi, Yin Tat Lee, and Lukas Wutschitz. Numerical composition of differential privacy. In Advances Neural Information Processing Systems, pages 11631-11642, 2021.
[28] Jiyan He, Xuechen Li, Da Yu, Huishuai Zhang, Janardhan Kulkarni, Yin Tat Lee, Arturs Backurs, Nenghai Yu, and Jiang Bian. Exploring the limits of differentially private deep learning with group-wise clipping. In The Eleventh International Conference on Learning Representutions, 2023.
[29] Kaiming He, Ross Girshick, and Piotr Dollar. Rethinking imagenet pre-training. In Proceedings of the IEEE/CVE International Conference on Computer Vision (ICCV), pages 4917-4926, 2019.
[30] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEFE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9729-979999 2020.
[31] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEFF/CVE conference on Computer Vision and Pattern Recognition (CVPR), pages 16000-16009, 2022.
12

---

[32] Elad Hoffer, Tal Ben-Nun, Itay Hubara, Niv Giladi, Torsten Hoefler, and Daniel Soudry. Augment your batch: Improving generalization through instance repetition. In Proceedings of the IEEE/CVE conference Computer Vision and Pattern Recognition (CVPR), pages 8126 8135, 2020.
[33] Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022.
[34] Florian A. HölzI, Daniel Rueckert, and Georgios Kaissis. Bridging the gap: Differentially private equivariant deep learning for medical image analysis. arXiv preprint arXiv:2209.04336, 2022.
[35] Florian A. Hölzl, Daniel Rueckert, and Georgios Kaissis. Equivariant differentially private deep learning. arXiv preprint arXiv:2301.13104 2023.
[36] Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proceedings of the 32nd International Conference on Machine Learning, pages 448-456. PMLR, 2015.
[37] Gautam Kamath, Argyris Mouzakis, Matthew Regehr, Vikrant Singhal, Thomas Steinke, and Jonathan Ullman. A staa-carrince trilemma for statistical estimation. arXiv preprint 3.Xi::3301.13334 2023.
[38] Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and improving the image quality of stylegan. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 8110-8119, 2020.
[39] Hirokatsu Kataoka, Kazushige Okayasu, Asato Matsumoto, Eisuke Yamagata, Ryosuke Yamada, Nakamasa Inoue, Akio Nakamura, and Yutaka Satoh. Pre-training without natural images. In Proceedings of the Asian Conference Computer Vision (ACCV), 2020.
[40] Helena Klause, Alexander Ziller, Daniel Rueckert, Kerstin Hammernik, and Georgios Kaissis. Differentially private training of residual networks with scale normalisation. arXiv preprint arXiv:2203.00324, 2022.
[41] Alex Krizhevsky. Learning multiple layers of features from tiny images, 2009.
[42] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems, 2012.
[43] Ananya Kumar, Aditi Raghunathan. Robbie Matthew Jones, Tengyu Ma, and Percy Liang. Fine- tuning can distort pretrained features and underperform out-of-distribution. In International Conference Learning Representations, 2022.
[44] Ann B Lee, David Mumford, and Jinggang Huang. Occlusion models for natural images: A statistical study of scale-invariant dead leaves model. International Journal of Computer Vision, 41:35 59, 2001.
[45] Tian Li, Manzil Zaheer, Sashank Reddi, and Virginia Smith. Private adaptive optimization with side information. In Proceedings of the 39th International Conference on Machine Learning. pages 19086-13105 PMLR, 2022.
[46] Xuechen Li, Daogao Liu, Tatsunori Hashimoto, Huseyin A Inan, Janardhan Kulkarni, YinTat Lee, and Abhradeep Guha Thakurta. When does differentially private learning not suffer in high dimensions? In Advances in Neural Information Processing Systems, pages 28616 16666,8,3,, 2022.
[47] Xuechen Li, Florian Tramèr, Percy Liang, and Tatsunori Hashimoto. Large language mod- els can be strong differentially private learners. In International Conference on Learning Representations, 2022.
[48] Wesley Maddox, Shuai Tang, Pablo Moreno, Andrew Gordon Wilson, and Andreas Damianou. Fast adaptation with linearized neural networks. In Proceedings of The 24th International Conference Artificial Intelligence and Statistics, pages 2737-2745. PMLR, 2021.
13

---

[49] Harsh Mehta, Walid Krichene, Abhradeep Guha Thakurta, Alexey Kurakin, and Ashok Cutkosky, Differentially private image classification from features. Transactions on Machine Learning Research, 2023. ISSN 2835-8856.
[50] Harsh Mehta, Abhradeep Guha Thakurta, Alexey Kurakin, and Ashok Cutkosky. Towards large scale transfer learning for differentially private image classification. Trunsactions on Machine Leurning Research, 2023. ISSN 2835-8856.
[51] Fangzhou Mu, Yingyu Liang, and Yin Li. Gradients as features for deep representation learning. In International Conference on Learning Representations, 2020.
[52] Milad Nasr, Saeed Mahloujifar, Xinyu Tang, Prateek Mittal, and Amir Houmansadr. Effec- tively using public data in privacy preserving machine learning. In Proceedings of the 40th International Conference on Machine Learning, pages 25718 257322. PMLR, 2023.
[53] Edouard Oyallon and Stéphane Mallat. Deep roto- trotstramslttion scattering for object classification. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition (CVPR), pages 2865-2873, 2015.
[54] Ashwinee Panda, Xinyu Tang. Vikash Sehwag, Saeed Mahloujifar, and Prateek Mittal. Dp-raft: differentially private recipe for accelerated fine-tuning. arXiv preprint rrXi:2212.004866 2022.
[55] Nicolas Papernot, Abhradeep Thakurta, Shuang Song, Steve Chien, and Úlfar Erlingsson. Tempered sigmoid activations for deep learning with differential privacy. Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 9312-9321, 2021.
[56] Francesco Pinto, Yaxi Hu, Fanny Yang, and Amartya Sanyal. Pillar: How to make semi- learning more effective. arXiv preprint arXiv:2306. 03962, 2023.
[57] Tom Sander, Pierre Stock, and Alexandre Sablayrolles. TAN without a burn: Scaling laws of DP-SGD. In Proceedings of the 40th International Conference Machine Learning, pages 29937-29949. PMLR, 2023.
[58] Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. In 2017 IEEE Symposium on Security and Privacy (SP), pages 3-18, 2017.
[59] Shuang Song, Kamalika Chaudhuri, and Anand D. Sarwate. Stochastic gradient descent with differentially private updates. In 2013 IEEE Global Conference on Signal and Information Processing, pages 245-248, 2013.
[60] Ziteng Sun, Ananda Theertha Suresh, and Aditya Krishna Menon. The importance of feature preprocessing for differentially private linear optimization. arXiv preprint arXiv:2307.11 106, 2023.
[61] Florian Tramer and Dan Boneh. Differentially private learning needs better features (or much more data). In International Conference on Learning Representations, 2021.
[62] Florian Tramèr, Gautam Kamath, and Nicholas Carlini. Considerations for differentially private learning with large-scale scale public pretraining. arXiv preprint arXiv:2212.06447,, 2022.
[63] Chendi Wang, Yuqing Zhu, Weijie J Su, and Yu-Xiang Wang. Neural collapse meets dif- ferential privacy: Curious behaviors of noisySGD with near-perfect representation learn- ing. In Conference on Parsimony and Learning (Recent Spotlight Track), 2024. URL htttps://openreee
[64] Tongzhou Wang and Phillip Isola. Understanding contrastive representation learning through alignment and uniformity on the hypersphere. In Proceedings of the 37th International Confer- ence on Machine Learning, pages (192999939 PMLR, 2020.
[65] Jiancheng Yang, Rui Shi, and Bingbing Ni. Medmnist classification decathlon: A lightweight automl benchmark for medical image analysis. In IEFE 18th International Symposium on Biomedical Imaging (ISBI), pages 191-195, 2021.
14

---

[66] Jiancheng Yang, Rui Shi. Donglai Wei, Zequan Liu, Lin Zhao, Bilian Ke, Hanspeter Pfister, and Bingbing Ni. Medmnist v2-a large-scale lightweight benchmark for 2d and 3d biomedical image classification. Scientific Data, 10(1):41, 2023.
[67] Ashkan Yousefpour, Igor Shilov, Alexandre Sablayrolles, Davide Testuggine, Karthik Prasad, Mani Malek, John Nguyen, Sayan Ghosh, Akash Bharadwaj. Jessica Zhao, Graham Cormode, and Ilya Mironov. Opacus: User-friendly differential privacy library in PyTorch. arXiv preprint arXiv:2109. 12298, 2021.
[68] Da Yu, Huishuai Zhang, Wei Chen, and Tie- Yan Liu. Do not let privacy overbill utility: Gradient embedding perturbation for private learning. In International Conference Learning Representations, 2021.
[69] Da Yu, Huishuai Zhang, Wei Chen, Jian Yin, and Tie- Yan Liu. Large scale private learning via low-rank reparametrization. In Proceedings of the 38th International Conference on Machine Learning, pages
PMLR, 2021.
[70] Da Yu, Saurabh Naik, Arturs Backurs, Sivakanth Gopi, Huseyin A Inan, Gautam Kamath, Janardhan Kulkarni, Yin Tat Lee, Andre Manoel, Lukas Wutschitz, Sergey Yekhanin, and Huishuai Zhang. Differentially private fine-tuning of language models. In International Conference on Learning Representations, 2022.
[71] Yaodong Yu, Maziar Sanjabi, Yi Ma, Kamalika Chaudhuri, and Chuan Guo. Vip: A differentially private foundation model for computer vision. arXiv preprint 23X1v:330608844 2023.
15

---

A Ablation study for Phase
We present our main results by using the WRN-16-4 model pretrained on Style-GAN dataset by Baradad et al. [5] with representation learning 644 for section, we provide more ablation study on the different synthetic data and different representation leamning methods in Phase I. We use CIFARIO for the evaluation.
In this
A.1 Results different synthetic data
number of synthetic datasets have been proposed by prior work. Baradad et al. [5] consider a family of synthetic datasets generated by random processes, and report that the best synthetic dataset in terms of downstream performance generated by an untrained StyleGAN with a specific initialization, denoted as StyleGAN --rreeeedd...d Based on this prior work, we also use StyleGAN-Orientee
as our main synthetic dataset throughout the main body of the paper. We also considered the Shaders dataset proposed in Baradad al. [6] for ImageNet experiments. In this subsection choices of synthetic datasets and find that multiple synthetic datasets can provide good performance. That is, our results and analysis extend beyond EyyeiGAN-rreented and can be applied to future proposed synthetic datasets.
consider different
We first use the feature extractor checkpoin provided by Baradad et al. [S] pretrained on different synthetic datasets (Please refer Baradad al. 5 for the full description of these datasets).
Note that Baradad et al. 55] uses a small Alex 42 as feature extractor, which includes Batch- Norm (36. We freeze the feature encoder and only train the last linear model, therefore the feature extractor does not break our differential privacy guarantee. When we use the feature extractor in the main body we use WideResNet without BatchNorm.
Tab. 10 summarizes the results on different synthetic datasets. As in Baradad et al. [5]. the best synthetic dataset is StyleGAN -ooreneedd. However, the worst-performing synthetic dataset (Dead leaves) performs similarly to the best prior work [5]. Tab. I0 suggests that one potential future direction of DP-RandB is better synthetic data.
Table 10: Test accuracy (%) of private linear probing training for 100 steps, that is the same as with WRN-16-4) on a small AlexNet trained on different synthetic images. StyleGAN-Oriented achieves the best performance. Note that all these images are generated without access to real- world images. Evaluation on CIFARIO task.
Dead leaves
Stat Color+WMM
Untrained StyleGAN
Feature Vis Dead leaves
textures
Sparse
High freq Oriented
Accuracy
59.92
64.59
64.34
64.08
67.79
59.32
A.2 Results on different representation learning methods
In Tab. TT we compare the representation learning method of Wang and Isola 64 (results already in the main body) to MoCo (30 (we use default hyperparameters in the official repository) for use in Phase I. For Phase II and III, we use the same hyperparameters as in Tab, 13
Similar to the main results, DP-RandP achieves significant improvements compared to baseline [15 We find that using either contrastive learning methods [64, 30 can achieve 72% accuracy at 1.
Also, Tab. II shows that DP-RandB consistently improves upon DP-RandP w/o Phase II when using either of Wang and Isola (64] or He et al. [30 Phase I. We note that while DP-RandP is robust to the two representation learning choices of for Phase there is small gap between the two methods as & increases. This suggests a future direction for further improving our method with principled choice of contrastive learning method for Phase I.
httgs//ggitibbocommmmmmaaaadlale with
with noise
16

---

Table 11: Test accuracy (%) of different representation learning methods in Phase Evaluation on 10 task.
Method for Phase
Phases
1
4
8
DP-RandP
72.32
77.25
79.99
81.88
84.01
85.26
91.69
Wang and Isola 64
DP-RandP w/o Phase
69.03
75.31
78.44
80.56
82.96
84.45
91.69
DP-RandP
72.79
76.26
78.32
79.78
$1.64
82.84
90.84
MoCo 30
DP-RandP w/o Phase II
69.48
73.82
76.60
78.89
80.56
82.64
90.84
Phase III only (De et al.
15
56.8
64.9
69.2
71.9
71.0
79.5
88.9
B Ablation study on model architecture
We present DP-RandP on CIFARIO with experiments with WRN-16-4 in the main body and here we also present the WRN-40-4 result on CIFARIO in Tab. 12 The result has a similar trend as De et al. [1]. WRN-40-4 can achieve better utility with more parameters. For example, at E = 1, WRN-40-4 has 0.83% increase compared to WRN-16-4. However, training WRN-40-4 model takes longer time. Training a WRN-16-4 for 875 steps takes 5.5 hours while the same amount of steps would take 12 hours for WRN-40-44 Given the utility improvement is within 2% improvement by changing from WRN-16-4 to WRN-40-4, we use WRN-16-4 to demonstrate the effectiveness of DP-RandP for the main experiments.
Table 12: Ablation study
WRN- and WRN-40-4 on CIFARIO.
Method
Model
E
3 5
6
E
8
WRN-1664
69.03
75.31
78.44
80.56
82.90
84.45
DP-RandP w/o Phase II
WRN-40-4
69.45
76.63
79.64
82.20
84.51
85.57
WRN-16-4
72.32
77.25
79.99
81.88
84.01
85.26
DP-RandP
WRN-40-4
73.15
77.53
80.93
82.83
85.17
86.12
C
Experimental details
We use the Opacus library 667 for the DP-SGD inplementation. Our experiments are based on the open-source code"oof of Sander al. (57) and Baradad al. [5). We also provide our code For the noise multiplier given sampling rate and total step size, a is precomputed according to privacy loss distribution accounting as implemented in Gopi al. [27] with epserror= 0.01 and rounded up the precision of 0.1 to ensure that we do not underestimate the privacy loss.
Hyperparameeers.. Tab. 13 14 and 15 summarize the hyperparameters for DP-RandP on CIFARIO, CIFAR100 and DermaMNIS' IST respectively. We use the same total steps and batch size for CIFARIO by following De et al. [15] We also use the same hyperparameters of batch size and steps for CIFAR100. For DermaMNIST, we use batch size 1024 because Hölzl et al. [34] follow Klause et al. and Klause et
40
use batch size 1024.
Table 13: Hyperparameters for DP-RandP on CIFARIO. Batch size is 4096 Augmult is 16, SGD optimizer with momentum 0 and no weight decay.
2
3
6
8
Total Steps
875
1125
1593
1687
1843
2468
O Steps in Phase II
9.3
5.6
4.7
3.8
2.9
2.6
96
96
96
96
96
96
LR in Phase
15
15
15
15
15
LR in Phase III
0.4
1
1
1.2
1.2
1.6
httgss/gghhbbboom/ffseebotteseee and https//githbb.co///maaaaaaaaaaat wwtpraaaaiiiniittttoiotooooo noise
17

---

Table 14: Hyperparameters for DP-RandP on CIFARI00. Batch size is 4096 and Augmult is 16, SGD optimizer with momentum and no weight decay.
3
6
8
Total Steps
1593
1687
1843
2468
4.7
3.8
2.9
2.6
Steps in Phase II
96
96
96
96
LR in Phase
25
25
25
LR in Phase III
1.4
2
2.2
1.8
Table 15: Hyperparameters for DP-RandP DermaMNIST. Batch size is 1024 and Augmult is 16, SGD optimizer with momentum and no weight decay.
7.42
Total Steps
600
800
800
13.6
4.6
2.8
Steps in Phase II
48
48
48
LR in Phase II
2.8
LR in Phase III
0.2
1
1.2
D Additional results on DermaMNIST
We follow Hölzl et al. [4] and report the validation accuracy of DermaMNIST in Tab. Here
also report the test accuracy in Tab. [6 and we can see DP-RandP outperforms the DP-SGD baseline.
Table 16: Test accuracy (%) of DP-RandP on DermaMNIST.
Method
c=4 4
$
7.42
8 = 00
DP-SGD we evaluated
68.34..28
71.08 71.08p.51
72.58p.19
76.16
DP-RandP
71.190.28
73.68u.24
75.04p.25
79.70
E
Privacy allocation method
We give general privacy budget allocation strategy in Sec. 3.3 In this section, give detailed description of our privacy allocation strategy, the privacy ratio for CIFARIO main results, and more results on different
Our privacy allocation strategy. Given total number of steps N, we use the first N₁ steps to train the head classifier for Phase II, and use the remaining N₂ N N₁ steps to train the full network for Phase III. We follow Panda et al. [54] (that suggests 100 steps for linear probing) and set Ni 96 steps (this is closest to 100 steps and equals 8 epochs as each epoch contains 12 steps) in our experiment. For the x-axis in Fig. 6 we use PLD accounting as implemented in Gopi et al. 27 to calculate by calculating the privacy cost of N₃ steps and get ? s as the x-axis. Although is known that E does not increase linearly with N1, Et is monotonically increasing with N₁ and therefore can this method to compute the privacy ratio of Phase II. The N1 steps in Fig. 6 include [0, 12, 24, 36, 48, 96, 144, 192, 240. 288, 336, 384, 432, 480, 528, 576, 624, 672, 720, 768, 816, 864, 875] with N 875.
Privacy ratio for CIFARIO main results. We visualize our privacy allocation strategy on CIFARIO in Fig. that is consistent with this strategy.
Additional experimental results on different &
We provide more results on different privacy
allocations for different from Tab, 17 to Tab 21 Our main results in Tab, in the main paper
18

---

Tracton of mrvecy co.dpptt Aesteed to Phonee
0.32
0.30
0.29
0.24
0.20
0.38
The fraction of total privacy budget allotted to Phase II (51/s) as function of total privacy budget e. As increases, the ratio $ decreases.
Figure 7:
produced by setting the number of epochs in Phase II to 8. From Tab. 17 to Tab 21 we can see that our privacy splitting strategy is robust several choices of the number of epochs in Phase II (e.g., 8, 12, 16) in the evaluated privacy range. Furthermore, our privacy splitting strategy is better than allocating the entire privacy budget to either Phase (rightmost) or Phase III only (leftmost).
Table 17: Test accuracy (%) for CIFARIO = = 2. 1125 steps. 94 epochs in total.
Epochs for Phase
0
50
94
Accuracy (%)
75.30
76.31
77.03
77.35
76.74
77.05
77.10
75.06
63.10
Table 18: Test accuracy (%) for CIFARIO = 3. 1593 steps.
133 epochs in total.
Epochs for Phase II
12
16
20
50
100
133
Accuracy (%)
78.34
79.20
79.71
79.74
79.48
80.27
80.04
79.21
75.47
63.93
Table 19;
Test accuracy (%) for CIFAR1O
4. 1687 steps.
141 epochs in total.
Epochs for Phase
12
16
100
141
Accuracy (%)
80.63
81.24
81.76
81.46
81.61
82.01
81.56
81.36
78.07
64.41
Table 20: Test accuracy (%) for CIFAR10
6. 1843
steps.
154 epochs in total.
Epochs for Phase II
0
8
12
16
20
50
100
150
154
Accuracy (%)
82.93
83.73
84.04
83.75
83.86
84.17
83.94
83.41
81.13
65.43
64.89
Table 21: Test accuracy (%) for CIFARIO
8. 2468 steps.
206 epochs in total.
Epochs for Phase
0
8
12
20
50
150
200
Accuracy 84.37
84.84
85.43
85.17
85.21
85.37
85.08
85.08
83.84
82.74
69.17
64.85
Details for private linear probing on CIFAR10
For CIFARI0, we follow Baradad et al. 5 and use the alignment and uniformity loss proposed in Wang and Isola [4 to pretrain feature extractor WRN-16-4 on StyleGAN-oriented dataset. Also we follow Baradad et al. [5] and use the third to last layer and the dimension of this layer is 4096.
19

---

Note that, the right extreme in Fig. iis slightly higher than 60% at E 1. while the linear probing result in Tab. 4 is 78% at E 1. This is because representation learning [64 IT usually adds additional representation layers for representation learning and may keep it for linear probing. As mentioned earlier, we follow Baradad al. [5] and use the third last layer and the dimension of this layer is 4096. For Fig. 6 we keep the exact same architecture De et al. [15 for fair comparison where we did not use such embedding layers.
Tab. 22 summarize the hyperparameter for DP-RandP without Phase III on CIFARIO in Sec 3.3
Table 22: Hyperparameterss for linear probing CIFARIO experiments. Other hyperparameters include full batch size, SGD optimizer with momemtum 0.9, 100 steps, no augmentation multiplicity.
E
0.1
0.2
0.5
1
2
3
0
339
171
72
38
21
14
11
8
LR
0.8
2.2
5.5
10
20
40
40
60
60
G Details for ImageNet experiments
As discussed Sec. 3.3 we achieve new SOTA on ImageNet with additional designs for private linear probing. We first present the technical details of our method, which combines principled feature extraction and private feature preprocessing approach. We then include the hyperparameters for our experiments on ImageNet.
G.1 Method
Our modifications on private linear probing include principled feature extraction and a private feature preprocessing approach. We qualify that neither of these steps novel; feature extraction variants and feature preprocessing are very standard feature extraction pipelines.
Pretraining. We pretrained ViT-base [19] model by Yu et al. 771), that is pretrained on the Shaders-2Ik dataset [6] using MAE BT. We use different pretraining dataset here than for the simpler datasets, because models pretrained on Shaders-21k are observed to outperform those pretrained on StyleGAN 66. Furthermore, based on our initial experiments models pretrained on StyleGAN with MoCo 60 do not perform well ImageNet fine-tuning, only reaching = 33%.
Modifying the LP-FT recipe. For the results on simpler datasets, we present a combination of linear probing and full fine-tuning (LP-FT [43). However, we note that for more complex datasets, it is more necessary to devote privacy budget to full fine-tuning because adapting the features learned from random priors to private datasets is more challenging. Based on the increase in the best values of E from CIFARIO and CIFAR100, it seems likely that full DP fine-tuning is necessary to adapt the pretrained features to ImageNet. Unfortunately, we lack the computational resources fine-tune ViT on ImageNet with DP as prior works 15 57) have noted that it may require hundreds or thousands of GPU-hours. Instead, propose a hybrid combination of linear probing and full fine-tuning via principled feature extraction that is computationally efficient, running under 10 hours on a single A100, and also obtains better performance than the prior SOTA [57]. We emphasize that these modifications are more for computational efficiency; if we had enough compute to do full fine-tuning of the ViT ImageNet with sufficiently large batch size, our original DP-Randl will still work.
Standard feature extraction. Standard CNNs and ResNets iteratively refine the representation of the image at each layer, and the best representation of the image is produced the penultimate layer. The head classifier at the end of the network learns a mapping between this representation and classes. slightly different; each block learns different representation. Nonetheless, the SOTA DP fine-tume tening approaches that use pretrained ViTs as feature extractors still just use the representation from the penultimate layer input to the linear layer (54). When we this approach for linear probing, our best result does not exceed 33.2%.
Vision transformers
Intuition behind principled feature extraction. The intuition behind this approach is the same the intuition a long line of fine-tuning approaches [48] ST 23 and we do not claim any novelty for Given sufficiently good pretrained initialization for the network (2),, the domain adaptation should
20

---

have low intrinsic rank such that the adaptation from the pretrained weights to the fine-tuning weights can be modeled in a lower-dimensional subspace than that of the full model parameters [33] Results have shown that linearized approximation of fine-tuning can obtain competitive performance. If we add another level of approximation, if we learn a linearization of the intermediate representations of the VIT, we can model the linearization of fine-tuning.
Principled feature extraction. The first challenge is the dimensionality of the intermediate representations. Although it may not seem large upon initial inspection, as ViTs may only have a representation size of 768, we actually want the representation before the pooling. The representation of the input after each block in the ViT has both a temporal and feature dimension, so we pool over the temporal dimension to gather feature map of size (4, feature size). One alternative option here is to actually learn first what weights should be used for the final linear layer, as in Evci et al. [23] We can privatize this via the exponential mechanism. Initial results indicate this may be a very interesting direction for future work. However, this approach has quite high computational cost, so we do not use it for the sake of reproducibility. Instead, we stride the average pooling such that the representations at each block are 4x larger, and then we concatenate together the block-wise representations so that the final representation size is 4 x num_ blocks larger than it is in the standard feature extraction approach. For a ViT-base, blocklocks representation size and the feature size for each image is 36864.
12 so this is 48x larger for final
Feature normalization. The first step in feature preprocessing is feature normalization. We normalize the feature vectors to a fixed norm of C' by the transformation below
o T; C
| 112
We treat C' as fixed constant, and hence this normalization step doesn't result in any privacy loss about the dataset. We pick C = 50 because the representation multiplier from the principled feature extraction step is 48. Next we center the feature vectors around their mean X; I; |D| ΣjED Zj., which requires private mean estimation. That is, the input to the training method will just be the difference between the feature and the noisy feature mean.
Private mean estimation. Now we introduce the motivation behind the feature normalization; so that we can do private mean estimation in high dimensions without prohibitive error rates. Given that all the feature vectors have the same dimension and fixed norm, the optimal error rate for private mean estimation will be obtained via the Gaussian mechanism. That is. we first compute the true mean and then add Gaussian noise scaled the & sensitivity of the mean. The sensitivity of the mean is C/N; adding or removing any datapoint can change the l2 sensitivity by at most C, and there are N datapoints (for ImageNet, N 1281167). We do a hyperparameter search here to find the best amount of the overall privacy budget to dedicate to this step, which we can do efficiently by saving the true mean and then adding noise for each value of we consider. We find that a very noisy estimate is sufficient, because the noise is added to the mean vector and then used to normalize all the features such that all datapoints have the same noise. This correlated noise is highly tolerable for our approach, in line with concurrent work that indicates correlated noise has much less accuracy degradation [
Related work on private feature preprocessing. One concurrent work (60 conducts a theoretical analysis that feature preprocessing provably reduces the error rate of DP linear regression and provides experiments when finetuning a model that is pretrained on ImaageNet (that is, the regime of Tab. 7 Another concurrent work [63] conducts theoretical analysis that feature preprocessing provably reduces the error rate of DP-GD from extracted features by connecting to the neural collapse regime and provides experimental validation when finetuning model that is pretrained on ImageNet. These concurrent works provide intuition behind the success of our private feature preprocessing. although we note that neither theoretical guarantee is actually applicable to our setting because we use an entirely different method (pretraining on synthetic data, principled feature extraction, and private feature preprocessing). Our method therefore validates the theory of these concurrent works by applying private feature preprocessing in conjunction with principled feature extraction to do private linear probing on one of the most challenging datasets for DP image classification, achieving new SOTA for methods that do not use public data.
21

---

G.2 Results
We achieve 39.39% accuracy at E = 8, and the previous SOTA [57] 39.2% at = 8. Although this improvement minor, we note that we have not been able reproduce the results in Sander et al. 157 using their provided code, who themselves were not able to reproduce the result from De al.
15 that they compare to. This reproducibility gap can be attributed to the high variance of DP training. which itself is difficult to mitigate because of the enormous computational cost required to get SOTA results on ImageNet privately. We hope that by providing our code we can bridge this gap.
We include hyperparameters of our private linear probing results on ImageNet in Tab. 23 Tab. 23 di for the private mean estimation step and d2 for DP-SGD. Because batch, we can use the composition theorem in Gaussian differential privacy [17] (Theorem 2.7 and Corollary 3.3) to compute the privacy loss. With T steps in DP-SGD) our private linear probing is equivalent to a one-step Gaussian mechanism with noise multiplier a, where
using the full
+
T
We can then compute (c, 6)-DP by computing the privacy curve of Gaussian mechanism 3
increases, the total step size will also increase to achieve better performance. Therefore Tab, 23 we use T 100 for
According to Panda et al. [54]
and = 200 for = = 8. In addition to Tab. 23 we also provide a few more observations during the hyperparameter search. T 100 for 8 also achieves compelling result that is close to 39%. When we fix E = 8, as we increase T, the performance improves. Further increasing T would continue improve the performance. As the number of steps T increases, the corresponding optimal learning rate would decrease. This is consistent with previous work [54]
We note that Sander al. (57 also tried training the ViT model on ImageNet but concluded that does not perform as well as ResNet. Our explanation for this is that ViT requires pretraining data because the architecture does not encode any natural prior, whereas CNNs naturally have a prior. The nature of convolutional filters biases CNNs to extract features with spatial locality. As observe in Sec. [2, the impact of pretraining data is mostly at the initialization by giving the model prior, so stands to reason that the missing piece in utilizing ViT for DP training on ImageNet is learning random prior from Shaders-211 (6)
Table 23: Hyperparameters for linear probing ImageNet. 0 is for private mean estimation and d2 is for DP-SGD. Other hyperparameters include batch size = full, SGD optimizer with momentum 0.9. For the linear layer, bias False and zero initialization. We do not employ any additional regularization or learning rate schedule.
Steps T
01
02
learning rate
8
100
71
43
3
200
14
9.33
10
H Two-stage training with private data
After pretraining with synthetic data, DP-RandP first training the classifier head (Phase II) and then tuning all hyperparameter (Phase III). Tab. [4 summarizes the results in the main paper and presents the comparison of our full DP-RandP DP-RandP without Phase II and the baseline [15]. Note that DP-RandP without Phase III not included in Tab. 24 as training the linear layer only has diminishing returns: the noi-private baseline of DP-Red DP-RandP w/o Phase III is 74.05%, which is worse than Ra-RandP w/o Phase II at 2. Tab. 24 shows the importance of combining both Phase II and Phase III in DP-RandP
22

---

Table 24: The importance of phases in DP-RandP. Evaluation of test accuracy (%) on CIFARIO.
Phases
= 1
2
4
= 6
=
8
DP-RandP
72.32
77.25
79.99
81.88
84.01
85.26
DP-RandP w/o Phase II
69.03
75.31
78.44
80.56
82.96
84.45
Phase III only (De al. [1]
56.8
64.9
69.2
71.9
71.0
79.5
Our DP-RandP uses synthetic data for pretraining to give warm initialization for two-stage training with private data. Tab. 25 shows the two-stage training with private data with random initialization. As noted in Tab. 13 we use a = 9.3 for of 875 steps (equals to 73 epochs). This is because we only round a to 0. and we ensure the we use in Tab. [3ill will not exceed the designed privacy bound, As we add less noise compared et al. [15], the baseline result, i.c., directly updating the full parameters during training (0 epoch for Phase II,, 57.46.
while De et al. [5] use 10 for the same number
Table 25: Test accuracy (%) of two-stage training with private data by random initialization on CIFARIO.
Epochs for Phase II
0
1
2
4
8
12
16
20
50
73
Accuracy (%)
57.46
57.01
56.27
55.04
54.45
54.09
53.15
52.26
47.00
25.80
Std.
0.48
0.69
0.57
0.90
0.80
0.41
0.91
0.97
1.53
2.08
Tab. 25 shows that when the feature extractor is randomly initialized, if we train the head classifier with more steps while keeping the same total steps, the performance will decrease compared to directly finetuning the whole network. Our analysis this is that, the feature extractor is not encoded with any prior and it is better to update the full network in the whole training procedure to learn more information.
23


Reviews:
[{"review_id": "NzCQdoRT2l", "reviewer": "Reviewer_t5fK", "summary": "The paper considers the problem of training ML models in a privacy-preserving way, focusing primarily on image classification. The main observation in this paper is that synthetic data learned from random processes can be used to learn useful visual priors which can be extremely useful for private training. Building on this, the authors propose a three phase approach for privately training ML models: 1. Learn a prior (feature extractor) from synthetic images generated by random processes, 2. Train the head classifier with frozen features with privacy parameter epsilon_1, and 3. End-to-end training using the remaining privacy budget epsilon_2. The authors provide an extensive empirical evaluation of this method over three different datasets for a wide range of privacy parameters, demonstrating that their methods significantly improve over state-of-the-art results for all privacy regimes. For example, for CIFAR10 with epsilon = 1, they obtain 72% accuracy, significantly improving over the SOTA (60%). \n", "questions": "See above.", "limitations": "See above.", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "\nAs written above, the paper has some clear and strong empirical improvements in the performance of private training for image classification. Moreover, the idea of using priors learned from synthetic data is interesting and turns out to be extremely useful for private training as demonstrated in the paper. This techniques could also be useful for private training in other domains.", "weaknesses": "In general, the paper has strong results, and aside from some minor comments (see below), my main critique is that the paper explains the algorithm too quickly without too much details, and starts the empirical evaluations too early. I would recommend the authors to spend more time in explaining their algorithms, and giving more background for phase 1 which is basically the main component of the algorithm. Also, the authors should give enough details for future readers to be able to replicate the results; for example, which version of DP-SGD is used? What is the batch size? Does it use momentum?\n\nMore comments:\n\n1. While the methods in the paper improve over SOTA for private training, they don’t achieve the same performance as private training with public data. I would suggest that the authors also compare their methods with methods that use a public dataset for pretaining (the authors currently do this only for epsilon = 1).\n\n2. The idea of having both phase 2 and 3 is interesting and it would be useful to see the improvements of that when combined with pretraining using public data.\n\n3. It would be useful to provide further evidence for the importance of combining both the second and third approached in the algorithm: one possibility is to compare the 3-phased approach with an approach that uses phase 1+2 or phase 1+3. This has been done in separate tables in the paper but providing the performance of all three approaches in the same plot would help better illustrate the advantages of all three phases.\n\n4. Privacy splitting:\n    - The privacy splitting strategy used in the paper is not completely clear and should be explained in more detail.\n\n    - The paper only shows how privacy splitting affects the performance for small values of eps=1 in Figure 6. It would be useful to provide a similar figure for larger values of epsilon, to show the advantages of phase 3 over phase 2 for such epsilons, and to show how robust is this method to privacy splitting.\n\n5. Can this approach be applied beyond image classification?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We first want to thank the reviewer for appreciating both the novelty and empirical success of our method, and furthermore the potential application to other domains.\n\n1. “Presentation and experiment details.” \n\nFor the experimental set-up, we use the DP-SGD introduced by Abadi et al.[11]. We use the opacus library for DP-SGD and our code is built based on the codebase by Sander et al.[12]. We mostly follow the hyperparameters suggested by De et al. [13] (batch size = 4096, momentum=0). We include our hyperparameters in Appendix C. We have open sourced the code in a comment to the ACs as per the NeurIPS rebuttal guidelines. We will improve the explanation of the algorithm and background in Phase I in the revised version as suggested. \n\n2. “Comparison to private training with public data”.\n\nPlease see CR#1 where we provide a quantitative comparison to methods that use public data.\n\n3. “Phase II and Phase III for public data setting.” \n\nGiven limited rebuttal time (more time is needed on the task with public data for a fair comparison, which is not studied in this submission), we did not have the chance to conduct experiments in Phase II and Phase III for public data. Here we briefly discuss our intuition for why Phase II and Phase III should also help for the public data setting. In the non-private scenario, self-supervised learning has already shown to be effective for large amounts of data [14] (especially when no label information is available). Table 5 in Yu et al. [15] shows that using public data with random labels to pretrain the model can achieve better utility for DP-SGD compared to the ground truth label, which suggests that the feature information in the public data is more important. Our Phase I is trained with feature information and does not need label information. Recent works [2,3,16] show that pretraining the model on the large public data and finetuning a linear layer with private data can achieve good performance on a wide range of tasks, which suggest that the linear layer is robust to the noise. For the pretrained model, Panda et al. [16] use the backbone model pretrained by Dosovitskiy et al.[14] (Mehta et al. [2,3] do not explicitly mention the pretraining method). We follow the intuition that the linear layer is robust to the noise, and we train our head classifier in Phase II and full network in Phase III. Considering the findings in [2,3,14,15,16], our framework should also help in the public data setting when there is a gap between linear probing and full finetuning. If the linear probing method is already pretty good as in [2,3], we also suggest directly applying linear probing.\n\n4. “It would be useful to provide further evidence for the importance of combining both the second and third approached in the algorithm”\n\nWe thank the reviewer for the suggestion. We will add Tab.17 in the revised paper. This table compares our full DP-RandP and DP-RandP without Phase II. We did not include DP-RandP w/o Phase III in this table as training the linear layer only has diminishing returns: the non-private baseline of DP-RandP w/o Phase III is 74.05%, which is worse than DP-RandP w/o Phase II at epsilon=2.\n\n5. “Privacy splitting method “\n\na. “Explanation for privacy splitting method” \n\nGiven a total number of steps N, we use the first N1 steps to train the head classifier for Phase II, and use the remaining N2 = N-N1 steps to train the full network for Phase III. We follow Panda et al.[16] (that suggests 100 steps for linear probing) and set N1 = 96 steps (this is closest to 100 steps and equals to 8 epochs as each epoch contains 12 steps) in our experiment. For the x-axis in Figure 6, we use PLD accounting as implemented in [24] to calculate $\\epsilon_1$ by calculating the privacy cost of N1 steps and get $\\epsilon_1$/$\\epsilon$ as the x-axis. Although it is known that $\\epsilon_1$ does not increase linearly with N1, $\\epsilon_1$ is monotonically increasing with N1 and therefore we can use this method to compute the privacy ratio of Phase II. We will include this explanation in the revised paper.\n\nb.”Experiments for different epsilons.” \n\nHere we provide more results for $\\varepsilon \\in [2, 3, 4, 6, 8]$ in Tab.19-23. Due to the limited time and computation constraints, we are not able to calculate error bars for these results. Our main results in Tab.1 in the main paper are produced by setting the number of epochs in Phase II to 8. From Tabs.19-23 we can see that our privacy splitting strategy is robust to several choices of the number of epochs in Phase II (e.g., {4, 8, 12, 16}) in the evaluated privacy range. Furthermore, our privacy splitting strategy is better than allocating the entire privacy budget to either Phase II or Phase III only. We will add these new results in our revised paper.\n\n6. “Other domains beyond image classification.”\n\nPlease see CR#2 for a discussion on extending our method to other data modalities.\n"}, {"review_id": "cAXAUNqY6F", "reviewer": "Reviewer_BUrr", "summary": "The authors propose using synthetic data to reduce the utility degradation induced by DP-SGD when training on image classification datasets. They suggest a 3 steps procedure: 1) pre-training on synthetic data, 2) DP training the linear head, 3) DP fine-tuning the full body. The procedure seems effective on the considered benchmarks and at various levels of $\\epsilon$. ", "questions": "- Did the authors consider the idea of augmenting their training set with fractals (e.g. [5], and not only black-and-white ones, but also coloured ones)\n- Did the authors consider applying other contrastive or self-supervised pre-training strategies to better leverage the synthetic data? It may be possible that different strategies can differently impact the accuracy of the linear probe (e.g. as shown in [4]) and may attain higher utility in Phase 3 of the training procedure.\n- Could you add error bars to the plots? (e.g. $+/- 2\\sigma$)\n\n[5] FractalDB, https://github.com/hirokatsukataoka16/FractalDB-Pretrained-ResNet-PyTorch", "limitations": "Properly Addressed.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Although the novelty is limited (the synthetic data was already designed by other works and the idea of pre-training to facilitate convergence in DP-SGD training has been explored), the observation this strategy is particularly effective in DP training is interesting and will be surely extremely useful for future research in DP training for image classification. \n- The paper is clearly written, the experiments are well explained.\n- The analyses carried out are interesting, and provide practitioners insights on how to adapt the proposed  training procedure to their setup \n", "weaknesses": "- The considered datasets are relatively simple. The method does not seem to scale well as the number of classes increases (see CIFAR-100 and ImageNet experiments). While this holds true for most DP training procedures, the ImageNet experiment might suggest the methodology has little or marginal utility in this case.  \n- While the comparison with ScatterNet and Equivariant-CNNs is particularly lengthy in the Related Works section, a comparison with methodologies leveraging public non-synthetic data (e.g. [1,2,3,4]) is missing. While such a comparison is unfair, and will show inferior performance of the proposed methodology, this does not diminish its utility. The purpose of such a comparison would be to show the gap that exists between using synthetically generated data (both data-driven or not) and real data, and would set the goal of bridging this gap with purely synthetic techniques in future works.  \n\nMinor:\n- The formatting of the captioning of when multiple subfigures are present on the same line does not render very well, I'd recommend improving it.\n\n[1] How to make semi-private learning more effective, https://openreview.net/pdf?id=vVXRNcltT6 \n\n[2] Differentially Private Image Classification from Features, https://arxiv.org/pdf/2211.13403.pdf\n\n[3] Large Scale Transfer Learning for Differentially Private Image Classification, https://arxiv.org/pdf/2205.02973.pdf\n\n[4] Differentially private diffusion models generate useful synthetic images, https://arxiv.org/pdf/2302.13861.pdf \n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the constructive feedback. We’re encouraged that the reviewer appreciates our empirical improvements, analysis and insights.\n\n1. ”...seem to not scale well…”\n\nWe first provide new and improved results on ImageNet; we achieve 39.35% accuracy at epsilon=8, and previous SOTA[12] is 39.2% at epsilon=8. First, as in our previous results, we consider a model pretrained on [25]; in this case the model is a ViT-base. If we do Phase II with extracted features, we achieve the same 33.2% at epsilon=8. We now explain the steps we took to improve our performance. We make two key modifications over the two-phase approach we use for other image tasks. We emphasize that these modifications are more for computational efficiency; if we had enough compute to do full fine-tuning of the ViT on ImageNet with sufficiently large batch size, our original two-phase approach will still work. \n\nOur first modification is to approximate full fine-tuning with linear probing on larger feature representations that we create by aggregating intermediate representations from the model. The representation of the input after each block in the ViT has both a temporal and feature dimension, so we pool over the temporal dimension to gather a feature map of size (4, feature size). The feature size is 768 for ViT-base with 12 blocks, so the representation size is 4\\*12\\*768, 48 times larger than the original representation we would have otherwise extracted. Our motivation is that because there is a large domain shift between [25] and ImageNet, the final representation may not be sufficient to learn the task. Fine-tuning is necessary to potentially lift up intermediate representations from earlier in the network. By doing linear probing on these much larger features, we can also take advantage of intermediate representations.\n\nThe second modification is to approximate the work of the normalization layer that we would update during full fine-tuning of the entire ViT, by manually normalizing the features. To do this we first normalize each feature vector to a fixed norm of 50, as the feature vector is 48 times larger than the original representation. There may be better normalization schemes, but we did not have enough time to tune this choice. We next privately estimate the mean over the entire ImageNet feature vector dataset, using the Gaussian mechanism with a privacy cost of epsilon=0.1, and subtract the private mean from all feature vectors. This is equivalent to doing non-private centering and then adding the same Gaussian noise to the entire dataset. This procedure can be thought of as a one-time approximation to the normalization layer that is known to speed up training by centering data.  \n\nThe hyperparameters we use for linear probing are: lr = 10, batch size = full, epochs = 200, optimizer = SGD, momentum = 0.9, bias = False, initialization = zero. We do not employ any additional regularization or learning rate schedule. \n\nWe note that [12] also tried training the ViT model on ImageNet but concluded that it does not perform as well as ResNet. Our explanation for this is that ViT requires pretraining data because the architecture does not encode any natural prior, whereas CNNs naturally have a prior. As we observe in the main paper, the impact of pretraining data is mostly at the initialization by giving the model a prior, so it stands to reason that the missing piece in utilizing ViT for DP training on ImageNet is pretraining on [25]. \n\nWe agree with the reviewer that there is still a large gap between the private result and non-private result on tasks with more classes like CIFAR-100 and ImageNet. This is due to the inherent difficulty of differentially-private training on these complex tasks (even outside the context of our innovations). Our work aims to close this gap by leveraging synthetic data to learn priors and utilizing our three phase framework. DP-RandP still achieves >10% improvements on CIFAR100 for the evaluated epsilons compared to the DP-SGD baseline. Future work in this direction to further improve the performance could include better synthetic data, model architecture, representation learning techniques for Phase I to learn better priors, better algorithms beyond our DP-RandP.\n\n2. “comparison with public data”\n\nPlease see CR#1.\n\n3. “format of caption”. \n\nWe will fix this in the revised paper.\n\n4. “augmentation with fractals”\n\nBaradad et al.[6] studied FractalDB[17] as a baseline and showed that StyleGAN-oriented is better than FractalDB for a range of tasks, though Baradad et al.[6] did not explicitly state the fractals is colored or black-and-white. Kataoka et al.[17] investigated the impact of colorization and achieved 93.1% accuracy (w/ color) vs. 92.9% accuracy (w/o color). Based on [16,17], possible improvements by current colored fractals for DP image classification tasks might be marginal. Future work on designing better fractal methods could be helpful in improving such tasks.\n\n5. “other contrastive learning“\n\nIn Tab.18 we compare the contrastive learning method of [18] (results already in the submission) to MoCo[19] (we use default hyperparameters in the official repo) for use in Phase I. For Phase II and III, we use same hyperparameters as in Tab.11. We will include Tab.18 in the revised paper.\n\nSimilar to the main results, DP-RandP achieves significant improvements compared to baseline[13]. We find that using either of [18,19] can achieve 72% accuracy at epsilon=1. Also, Tab.18 shows that DP-RandP consistently improves upon DP-RandP w/o Phase II when using either of [18,19] in Phase I. \nWe note that while DP-RandP is robust to the choice of [18,19] for Phase I, there is a small gap between the choice of [18,19] as epsilon increases. This suggests a future direction for further improving our method with a principled choice of contrastive learning method for Phase I.\n\n6. “...Error bars”.\n\nWe currently report std. in the tables. We will add error bars in the plots.\n"}, {"review_id": "o07fu6YzNT", "reviewer": "Reviewer_mcxP", "summary": "The paper focuses on the problem of improving the privacy-utility trade-offs for the foundational technique of DP-SGD. The authors explore gains from learning priors from images generated by random processes and transferring the priors to private data. They do this via designing DP-RandP, a three-phase approach that first learns priors, then training a head classifier on top of frozen features, and then training both the feature extractor and classifier. Through an empirical evaluation on CIFAR10, CIFAR100, and MedMNIST, they show that their method is able to achieve new SOTA accuracies when training from scratch.", "questions": "Since the novel technique has 2 new phases compared to prior works (e.g., De et al. [13]), it might be useful to a reader to understand what gains the first stage provides vs. gains provided by the second phase by itself? Although this is somewhat studied in section 3.3, I feel it might be useful to understand in the evaluation in section 3.2 (so DP-RandP can be compared with existing baselines, and with specific phases provided to those baselines).\n", "limitations": "Yes.", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1. The paper focuses on the important problem of improving the privacy-utility trade-offs for the foundational technique of DP-SGD.\n2. The paper is well-written, and the novel ideas are easy to understand.\n", "weaknesses": "Minor point:\n1. I think it might be cleaner to mention the gains in privacy cost in terms of the absolute gains in the parameter used to measure privacy. E.g., in Fig 2 caption, since $\\epsilon$ goes from 6 to 3, it might be easier to understand a factor of 2 gain? Stating a factor of 20 (which technically applies to the change in the outcome probabilities used in the defn. of DP) might be somewhat misleading (and overstated). Same comment for the gains stated in line 188.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the valuable feedback. We appreciate the reviewer recognizing our empirical improvements on the important problem.\n\n1. “privacy gain”.\n\nThanks for the suggestion. We will change the statement to reducing privacy cost from epsilon=6 to epsilon=3 in the revised paper (similar change to line 188).\n\n2. ”providing results for understanding the gains of three phases in section 3.2”.\n\nWe will add Tab.17 for CIFAR10 in the revised paper. The result on CIFAR10 is already in the paper and here we summarize them in a single table for better comparison. This table compares our full DP-RandP and DP-RandP without Phase II. We did not include DP-RandP w/o Phase III in this table as the linear layer has diminishing returns: the non-private baseline of DP-RandP w/o Phase III is 74.05%, which is worse than DP-RandP w/o Phase II at epsilon=2.\n\nIn addition, we have also tried DP-RandP w/o Phase I, which does not include the representation learning phase with synthetic data. The results of such a setting are worse than that in De et al.[13]. The explanation is that, when the model is randomly initialized, there is no additional utility gain to first fix the feature extractor and train the linear layer, as the feature extractor does not contain any useful information and it is better to update the full network in the whole training procedure.\n"}, {"review_id": "FRaFMQUCip", "reviewer": "Reviewer_dEgE", "summary": "This paper presents a method for synthesizing pre-training data for private fine-tuning without requiring access to any real data. The data generation process utilizes some fundamental properties of natural images that have been previously studied in fields outside of private learning. For example, natural images often exhibit a high degree of sparsity and contain oriented structures. After pre-training the model on the synthesized data, the authors employ a two-stage private learning framework. The first stage involves tuning only the linear classifier, while the second stage tunes all parameters.", "questions": "1. Is it easy to generate ‘free’ pre-training data for language or speech tasks? Language or speech data also exhibits some natural structures, e.g., the subject–verb–object word order. However, I’m not sure whether there are off-the-shelf algorithms that use such properties to generate data.\n\n2. I see the authors mark 'Reproducibility' as yes but the code is not uploaded. Is there a plan to make the code that reproduces all the main observations publicly available?", "limitations": "Limitations are discussed adequately.", "rating": 7, "confidence": 4, "soundness": 2, "presentation": 4, "contribution": 3, "strengths": "1. The concept of generating 'free' pre-training data is intriguing, and the methods employed in this paper appear reasonable. \n\n2. Experiments conducted on several vision datasets demonstrate that such synthesized data is important in private learning, particularly when the privacy budget is small.\n\n3. The authors introduce a two-stage approach for private learning, which is new to the private learning community and results in some improvements.\n\n4. The results are well-organized and easy to follow. \n", "weaknesses": "1. As shown in Figure 6, some improvements of the proposed algorithm are from the two-stage approach (Phase-II and Phase-III) for private learning. Can the two-stage approach improve the baseline? E.g., can the method in Hölzl et al., [27] get better results if it also uses Phase-II and Phase-III for private learning and set the hyperparameters accordingly?\n\n2. Kumar et al., 2022 [35] propose to first tune the classification head and then tune all parameters. The authors may want to discuss how their motivation can also explain the benefit of the two-stage approach, as a complementary to the discussion in Line 102 – Line 114.\n\nI am willing to increase my score if the authors can address my concerns.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the reviewer for the insightful feedback. We appreciate the reviewer recognizing our empirical improvements, as well as the importance of synthetic data and two-phase training.\n\n1. “Can the two-stage approach improve the baseline? E.g., can the method in Hölzl et al., [27]”\n\nWe incorporate DP-RandP with the baseline by De et al.[13] which uses the WideResNet architecture with GroupNorm layer and show that our DP-RandP can achieve significant improvement compared to De et al.[13]. Our DP-RandP is compatible with advancements in synthetic data, architecture, and representation learning. As discussed in Section 4, “we could use the architecture of Hölzl et al.[26,27] in our method and potentially enjoy the improvements of their method stacked on top of our own improvements by combining the feature priors”. As currently the source code of Hölzl et al.[20] is not available, we defer incorporation of our DP-RandP with their architecture design as future work. We look forward to investigating whether applying the equivariant neural network and our framework for learning from priors can further improve the utility once the authors release their code.\n\n2. “Discussion on Kumar et al.”\n\nAs discussed in Kumar et al.[21], the two-stage approach of first fine-tuning the classifier head and then tuning all hyperparameters has shown to be effective in previous studies[22,23]. Kumar et al.[21] show that such an approach is also effective for out-of-distribution (OOD) tasks. Their reasoning is that if the full parameters are updated using the in-distribution training data in the fine-tuning stage, then the full network would be more likely to overfit to the in-distribution data. This may lead to the loss of some general features learned in the pretrained stage and result in inferior performance on OOD data. They therefore utilize this two stage framework to solve this problem for OOD data. \n\nOur task of DP image classification is different from the OOD task in Kumar et al.[21] but still shares some common intuition. The training data and test data in our experiments follow the same distribution. While there is a large distribution shift between synthetic data generated by StyleGAN-oriented and real data in CIFAR10, training on the CIFAR10 training data should not cause utility drop in CIFAR10 test data. We conduct experiments and find that the two-stage pipeline would not significantly improve the performance compared to directly training the full parameters if no noise is added during training. After stage one, our feature extractor has learned some useful information while the classifier head has a random initialization. If we directly update the full network, too much noise is added to the full network, which may distort features learned in Phase I and lead to suboptimal performance. Panda et al [16] show that the linear head classifier is robust to noise and can be trained in less than 100 steps. We therefore first apply finetuning to the classifier head with only a few steps (96 steps for CIFAR10, which is a small privacy cost compared to the total privacy budget) and then tune all hyperparameters. We will add this discussion in the revised paper.\n\n3. ”pre-training data for other domain”\n\nPlease see CR#2 for a discussion on pre-training data for other data modalities.\n\n4. ”Reproducibility”.\n\nPlease see CR#3 for the anonymous code link.\n"}]

Author rebuttal:
We thank the reviewers for their constructive comments and suggestions. We’re encouraged that reviewers appreciate our empirical improvements, novelty, analysis, insights and potential impacts. We provide responses to common comments below. Tab.1-15 are in the current submission and Tab.16- 23 are included in the uploaded PDF. Due to space limitation, references are included as comment after common response.

1. “Comparison to private training with public data”.

We have a paragraph discussion on “DP with public data” in Section 4, where we briefly compare our method with Nasr et al. [1] at epsilon=1. Here we include the comparison of our method to previous works [1,2,3,4,5] by considering different public data settings and different epsilons in Tab.16 (quantitative comparison with [1,2,3]). We will add Tab.16 and discussion in our revised paper. 

Our DP-RandP is comparable with Nasr et al.[1], which in fact utilizes a limited amount of in-distribution data as public data for pre-training. This comparison supports the promise of our DP-RandP as mentioned in the paper “the prior learned from images generated from random processes can help as much as the prior learned from limited in-distribution public data.” 

There is also another line of work that leverages a large real-world public dataset to pretrain models [2,3,4]. Mehta et al. [2,3] can achieve > 95% accuracy even for epsilon=1 by pretraining on ImageNet. Pinto et al.[4] studied projecting the data into top-k principal components at epsilon<=1 and can achieve 81.3% accuracy at epsilon=0.1 using ResNet-50 pretrained on ImageNet as the feature extractor. Beside directly training image classification by DP-SGD, another direction is DP-trained generative models. The generated images can then be used for classification tasks without additional privacy costs. Recent work [5] showed that DP diffusion models can achieve high-quality images when pretrained on large public data like ImageNet. For example, [5] can achieve 88.8% classification accuracy for CIFAR10 at epsilon=10. We acknowledge that there is still a gap between our DP-RandP and these works (since they have access to real public data). Closing the gap between leveraging synthetic data and leveraging large-scale real public data is an interesting direction of future work (some possible ideas could be: better synthetic data, model architecture, representation learning algorithm for Phase I to learn better priors, and better algorithms beyond our DP-RandP).

2. ”Other domain beyond image classification”

We thank reviewers for this very interesting question! DP-RandP significantly enhances the state-of-the-art by utilizing the synthetic data generated from StyleGAN-oriented in image classification tasks. We believe that synthetic data can be generated with human knowledge in other domains as well to enhance differentially-private training. We provide a preliminary discussion on speech tasks and language modeling tasks below.

a. Speech tasks.

Speech data shares some similar properties to image data. The image data has a space domain and a frequency domain. The speech data has a time domain and a frequency domain. The synthetic data by Baradad et al.[6] is designed based on known characteristics of the frequency domain of images. It is also possible to generate the speech data in the frequency domain based on frequency knowledge of syllables for natural speech and then transform it back to the time domain. We could also incorporate the subject–verb–object word order in the time domain for synthesis as suggested by the reviewer dEgE. As for the training method, recent works by Chiu et al. [7] also showed that the self-supervised learning helps speech recognition tasks. These findings shed light on future applicability of our method in speech recognition tasks.

b. Language tasks.

As suggested by the reviewer dEgE, subject–verb–object word order could be designed for text synthesis. We could also encode the fact that synonyms should be close to each other in embeddings. Self-supervised learning methods such as BERT[8] and Roberta[9] are already widely studied in NLP tasks. We could generate several n-grams which do not have privacy risks for the language modeling tasks. Furthermore, Shi et al.[10] showed that first finetuning the model with private text removed corpus can help the followed differentially-private training. Although this work [10] utilizes more than synthetic data, we believe it provides insight into the feasibility and future applicability of our method in language tasks. 

3. “Reproducibility”.

We will open source our code. We include the link to anonymous codebase in a separate comment to Area Chair by following NeurIPS rebuttal guideline.


Meta-review: This paper proposes a novel differentially private training method that substantially improves the state-of-the-art for DP training on widely researched datasets such as CIFAR-10.

There is an agreement among the reviewers to accept this paper and recommend it for a spotlight.
Decision: Accept (spotlight)