SlowGuess/NeurIPS_2023_OCR huggingface dataset:
B Ablation study on model architecture
We present DP-RandP on CIFAR10 with experiments with WRN-16-4 in the main body and here we also present the WRN-40-4 result on CIFAR10 in Tab. 12. The result has a similar trend as De et al. [15], WRN-40-4 can achieve better utility with more parameters. For example, at  $\varepsilon = 1$ , WRN-40-4 has a  $0.83\%$  increase compared to WRN-16-4. However, training WRN-40-4 model takes a longer time. Training a WRN-16-4 for 875 steps takes 5.5 hours while the same amount of steps would take 12 hours for WRN-40-4. Given the utility improvement is within  $2\%$  improvement by changing from WRN-16-4 to WRN-40-4, we use WRN-16-4 to demonstrate the effectiveness of DP-RandP for the main experiments.
Table 12: Ablation study on WRN-16-4 and WRN-40-4 on CIFAR10.
Method	Model	ε = 1	ε = 2	ε = 3	ε = 4	ε = 6	ε = 8
DP-RandP w/o Phase II	WRN-16-4	69.03	75.31	78.44	80.56	82.90	84.45
WRN-40-4	69.45	76.63	79.64	82.20	84.51	85.57
DP-RandP	WRN-16-4	72.32	77.25	79.99	81.88	84.01	85.26
WRN-40-4	73.15	77.53	80.93	82.83	85.17	86.12
We use the Opacus library [67] for the DP-SGD implementation. Our experiments are based on the open-source code of Sander et al. [57] and Baradad et al. [5]. We also provide our code. For the noise multiplier  $\sigma$ , given sampling rate and total step size,  $\sigma$  is precomputed according to privacy loss distribution accounting as implemented in Gopi et al. [27] with epserror  $= 0.01$  and rounded up to the precision of 0.1 to ensure that we do not underestimate the privacy loss.
Hyperparameters. Tab. 13, 14 and 15 summarize the hyperparameters for DP-RandP on CIFAR10, CIFAR100 and DermaMNIST respectively. We use the same total steps and batch size for CIFAR10 by following De et al. [15]. We also use the same hyperparameters of batch size and steps for CIFAR100. For DermaMNIST, we use batch size 1024 because Holzl et al. [34] follow Klause et al. [40] and Klause et al. [40] use batch size 1024.
Table 13: Hyperparameters for DP-RandP on CIFAR10. Batch size is 4096 Augmult is 16, SGD optimizer with momentum 0 and no weight decay.
ε	1	2	3	4	6	8
Total Steps	875	1125	1593	1687	1843	2468
σ	9.3	5.6	4.7	3.8	2.9	2.6
Steps in Phase II	96	96	96	96	96	96
LR in Phase II	15	15	15	15	15	15
LR in Phase III	0.4	1	1	1.2	1.2	1.6
<sup>5</sup>https://github.com/facebookresearch/tan and https://github.com/mbaradad/learning_with_noise.
Table 14: Hyperparameters for DP-RandP on CIFAR100. Batch size is 4096 and Augmult is 16, SGD optimizer with momentum 0 and no weight decay.
ε	3	4	6	8
Total Steps	1593	1687	1843	2468
σ	4.7	3.8	2.9	2.6
Steps in Phase II	96	96	96	96
LR in Phase II	25	25	25	25
LR in Phase III	1.4	2	2.2	1.8
Table 15: Hyperparameters for DP-RandP on DermaMNIST. Batch size is 1024 and Augmult is 16, SGD optimizer with momentum 0 and no weight decay.
ε	1	4	7.42
Total Steps	600	800	800
σ	13.6	4.6	2.8
Steps in Phase II	48	48	48
LR in Phase II	2	2	2.8
LR in Phase III	0.2	1	1.2
D Additional results on DermaMNIST
We follow Holzl et al. [34] and report the validation accuracy of DermaMNIST in Tab. 3. Here we also report the test accuracy in Tab. 16 and we can see DP-RandP outperforms the DP-SGD baseline.
Table 16: Test accuracy (%) of DP-RandP on DermaMNIST.
Method	ε = 1	ε = 4	ε = 7.42	ε = ∞
DP-SGD we evaluated	68.340.28	71.080.51	72.580.19	76.16
DP-RandP	71.190.28	73.680.24	75.040.25	79.70
E Privacy allocation method
We give a general privacy budget allocation strategy in Sec. 3.3. In this section, we give a detailed description of our privacy allocation strategy, the privacy ratio for CIFAR10 main results, and more results on different  $\varepsilon$ .
Our privacy allocation strategy. Given a total number of steps  $N$ , we use the first  $N_{1}$  steps to train the head classifier for Phase II, and use the remaining  $N_{2} = N - N_{1}$  steps to train the full network for Phase III. We follow Panda et al. [54] (that suggests 100 steps for linear probing) and set  $N_{1} = 96$  steps (this is closest to 100 steps and equals 8 epochs as each epoch contains 12 steps) in our experiment. For the x-axis in Fig. 6, we use PLD accounting as implemented in Gopi et al. [27] to calculate  $\varepsilon_{1}$  by calculating the privacy cost of  $N_{1}$  steps and get  $\varepsilon_{1} / \varepsilon$  as the x-axis. Although it is known that  $\varepsilon_{1}$  does not increase linearly with  $N_{1}, \varepsilon_{1}$  is monotonically increasing with  $N_{1}$  and therefore we can use this method to compute the privacy ratio of Phase II. The  $N_{1}$  steps in Fig. 6 include [0, 12, 24, 36, 48, 96, 144, 192, 240, 288, 336, 384, 432, 480, 528, 576, 624, 672, 720, 768, 816, 864, 875] with  $N = 875$ .
Privacy ratio for CIFAR10 main results. We visualize our privacy allocation strategy on CIFAR10 in Fig. 7, that is consistent with this strategy.
Additional experimental results on different  $\varepsilon$ . We provide more results on different privacy allocations for different  $\varepsilon$  from Tab. 17 to Tab.21. Our main results in Tab. 1 in the main paper are
Figure 7: The fraction of total privacy budget allotted to Phase II  $(\varepsilon_1 / \varepsilon)$  as a function of total privacy budget  $\varepsilon$ . As  $\varepsilon$  increases, the ratio  $\varepsilon_1 / \varepsilon$  decreases.
produced by setting the number of epochs in Phase II to 8. From Tab. 17 to Tab.21 we can see that our privacy splitting strategy is robust to several choices of the number of epochs in Phase II (e.g., 4, 8, 12, 16) in the evaluated privacy range. Furthermore, our privacy splitting strategy is better than allocating the entire privacy budget to either Phase II (rightmost) or Phase III only (leftmost).
Table 17: Test accuracy (%) for CIFAR10  $\varepsilon = 2$ . 1125 steps. 94 epochs in total.
Epochs for Phase II	0	1	4	8	12	16	20	50	94
Accuracy (%)	75.30	76.31	77.03	77.35	76.74	77.05	77.10	75.06	63.10
Table 18: Test accuracy (%) for CIFAR10  $\varepsilon = 3$ . 1593 steps. 133 epochs in total.
Epochs for Phase II	0	1	4	8	12	16	20	50	100	133
Accuracy (%)	78.34	79.20	79.71	79.74	79.48	80.27	80.04	79.21	75.47	63.93
Table 19: Test accuracy (%) for CIFAR10  $\varepsilon = 4$ . 1687 steps. 141 epochs in total.
Epochs for Phase II	0	1	4	8	12	16	20	50	100	141
Accuracy (%)	80.63	81.24	81.76	81.46	81.61	82.01	81.56	81.36	78.07	64.41
Table 20: Test accuracy (%) for CIFAR10  $\varepsilon = 6$ . 1843 steps. 154 epochs in total.
Epochs for Phase II	0	1	4	8	12	16	20	50	100	150	154
Accuracy (%)	82.93	83.73	84.04	83.75	83.86	84.17	83.94	83.41	81.13	65.43	64.89
Table 21: Test accuracy (%) for CIFAR10  $\varepsilon  = {8.2468}$  steps. 206 epochs in total.
Epochs for Phase II	0	1	4	8	12	16	20	50	100	150	200	206
Accuracy (%)	84.37	84.84	85.43	85.17	85.21	85.37	85.08	85.08	83.84	82.74	69.17	64.85
F Details for private linear probing on CIFAR10
For CIFAR10, we follow Baradad et al. [5] and use the alignment and uniformity loss proposed in Wang and Isola [64] to pretrain a feature extractor WRN-16-4 on StyleGAN-oriented dataset. Also we follow Baradad et al. [5] and use the third to last layer and the dimension of this layer is 4096.
Note that, the right extreme in Fig. 6 is slightly higher than  $60\%$  at  $\varepsilon = 1$ , while the linear probing result in Tab. 4 is  $67.78\%$  at  $\varepsilon = 1$ . This is because representation learning [64, 11] usually adds additional representation layers for representation learning and may keep it for linear probing. As mentioned earlier, we follow Baradad et al. [5] and use the third to last layer and the dimension of this layer is 4096. For Fig. 6, we keep the exact same architecture as De et al. [15] for a fair comparison where we did not use such embedding layers.
Tab. 22 summarize the hyperparameter for DP-RandP without Phase III on CIFAR10 in Sec.3.3.
Table 22: Hyperparameters for linear probing CIFAR10 experiments. Other hyperparameters include full batch size, SGD optimizer with momentum 0.9, 100 steps, no augmentation multiplicity.
ε	0.1	0.2	0.5	1	2	3	4	6	8
σ	339	171	72	38	21	14	11	8	7
LR	0.8	2.2	5.5	10	20	40	40	60	60
G Details for ImageNet experiments
As discussed in Sec. 3.3, we achieve a new SOTA on ImageNet with additional designs for private linear probing. We first present the technical details of our method, which combines principled feature extraction and a private feature preprocessing approach. We then include the hyperparameters for our experiments on ImageNet.
G.1 Method
Our modifications on private linear probing include principled feature extraction and a private feature preprocessing approach. We qualify that neither of these steps is novel; feature extraction variants and feature preprocessing are very standard in feature extraction pipelines.
Pretraining. We use a pretrained ViT-base [19] model by Yu et al. [71], that is pretrained on the Shaders-21k dataset [6] using MAE [31]. We use a different pretraining dataset here than for the simpler datasets, because models pretrained on Shaders-21k are observed to outperform those pretrained on StyleGAN [6]. Furthermore, based on our initial experiments models pretrained on StyleGAN with MoCo [30] do not perform well on ImageNet fine-tuning, only reaching  $\approx 33\%$ .
Modifying the LP-FT recipe. For the results on simpler datasets, we present a combination of linear probing and full fine-tuning (LP-FT [43]). However, we note that for more complex datasets, it is more necessary to devote privacy budget to full fine-tuning because adapting the features learned from random priors to private datasets is more challenging. Based on the increase in the best values of  $\varepsilon_{1}$  from CIFAR10 and CIFAR100, it seems likely that full DP fine-tuning is necessary to adapt the pretrained features to ImageNet. Unfortunately, we lack the computational resources to fine-tune ViT on ImageNet with DP as prior works [15, 57] have noted that it may require hundreds or thousands of GPU-hours. Instead, we propose a hybrid combination of linear probing and full fine-tuning via principled feature extraction that is computationally efficient, running under 10 hours on a single A100, and also obtains better performance than the prior SOTA [57]. We emphasize that these modifications are more for computational efficiency; if we had enough compute to do full fine-tuning of the ViT on ImageNet with sufficiently large batch size, our original DP-RandP will still work.
Standard feature extraction. Standard CNNs and ResNets iteratively refine the representation of the image at each layer, and the best representation of the image is produced at the penultimate layer. The head classifier at the end of the network learns a mapping between this representation and classes. Vision transformers are slightly different; each block learns a different representation. Nonetheless, the SOTA DP fine-tuning approaches that use pretrained ViTs as feature extractors still just use the representation from the penultimate layer as input to the linear layer [54]. When we use this approach for linear probing, our best result does not exceed  $33.2\%$ .
Intuition behind principled feature extraction. The intuition behind this approach is the same as the intuition in a long line of fine-tuning approaches [48, 51, 23] and we do not claim any novelty for it. Given a sufficiently good pretrained initialization for the network [29], the domain adaptation should
have low intrinsic rank such that the adaptation from the pretrained weights to the fine-tuning weights can be modeled in a lower-dimensional subspace than that of the full model parameters [33]. Results have shown that a linearized approximation of fine-tuning can obtain competitive performance. If we add another level of approximation, if we learn a linearization of the intermediate representations of the ViT, we can model the linearization of fine-tuning.
Principled feature extraction. The first challenge is the dimensionality of the intermediate representations. Although it may not seem large upon initial inspection, as ViTs may only have a representation size of 768, we actually want the representation before the pooling. The representation of the input after each block in the ViT has both a temporal and feature dimension, so we pool over the temporal dimension to gather a feature map of size (4, feature size). One alternative option here is to actually learn first what weights should be used for the final linear layer, as in Evci et al. [23]. We can privatize this via the exponential mechanism. Initial results indicate this may be a very interesting direction for future work. However, this approach has a quite high computational cost, so we do not use it for the sake of reproducibility. Instead, we stride the average pooling such that the representations at each block are  $4 \times$  larger, and then we concatenate together the block-wise representations so that the final representation size is  $4 \times$  num_blocks larger than it is in the standard feature extraction approach. For a ViT-base, num_blocks = 12 so this is  $48 \times$  larger for a final representation size and the feature size for each image is 36864.
Feature normalization. The first step in feature preprocessing is feature normalization. We normalize the feature vectors to a fixed norm of  $C$  by the transformation below
$$
x _ {i} ^ {\prime} = \frac {x _ {i} \cdot C}{\| x _ {i} \| _ {2}}.
$$
We treat  $C$  as a fixed constant, and hence this normalization step doesn't result in any privacy loss about the dataset. We pick  $C = 50$  because the representation multiplier from the principled feature extraction step is 48. Next we center the feature vectors around their mean  $x_{i} = x_{i} - \frac{1}{|D|}\sum_{j\in D}x_{j}$ , which requires private mean estimation. That is, the input to the training method will just be the difference between the feature and the noisy feature mean.
Private mean estimation. Now we introduce the motivation behind the feature normalization; so that we can do private mean estimation in high dimensions without prohibitive error rates. Given that all the feature vectors have the same dimension and fixed norm, the optimal error rate for private mean estimation will be obtained via the Gaussian mechanism. That is, we first compute the true mean and then add Gaussian noise scaled to the  $\ell_2$  sensitivity of the mean. The sensitivity of the mean is  $C / N$ ; adding or removing any datapoint can change the  $\ell_2$  sensitivity by at most  $C$ , and there are  $N$  datapoints (for ImageNet,  $N = 1281167$ ). We do a hyperparameter search here to find the best amount of the overall privacy budget to dedicate to this step, which we can do efficiently by saving the true mean and then adding noise for each value of  $\varepsilon$  we consider. We find that a very noisy estimate is sufficient, because the noise is added to the mean vector and then used to normalize all the features such that all datapoints have the same noise. This correlated noise is highly tolerable for our approach, in line with concurrent work that indicates correlated noise has much less accuracy degradation [13].
Related work on private feature preprocessing. One concurrent work [60] conducts a theoretical analysis that feature preprocessing provably reduces the error rate of DP linear regression and provides experiments when finetuning a model that is pretrained on ImageNet (that is, the regime of Tab. 7. Another concurrent work [63] conducts theoretical analysis that feature preprocessing provably reduces the error rate of DP-GD from extracted features by connecting to the neural collapse regime and provides experimental validation when finetuning a model that is pretrained on ImageNet. These concurrent works provide intuition behind the success of our private feature preprocessing, although we note that neither theoretical guarantee is actually applicable to our setting because we use an entirely different method (pretraining on synthetic data, principled feature extraction, and private feature preprocessing). Our method therefore validates the theory of these concurrent works by applying private feature preprocessing in conjunction with principled feature extraction to do private linear probing on one of the most challenging datasets for DP image classification, achieving a new SOTA for methods that do not use public data.
We achieve  $39.39\%$  accuracy at  $\varepsilon = 8$ , and the previous SOTA [57] is  $39.2\%$  at  $\varepsilon = 8$ . Although this improvement is minor, we note that we have not been able to reproduce the results in Sander et al. [57] using their provided code, who themselves were not able to reproduce the result from De et al. [15] that they compare to. This reproducibility gap can be attributed to the high variance of DP training, which itself is difficult to mitigate because of the enormous computational cost required to get SOTA results on ImageNet privately. We hope that by providing our code we can bridge this gap.
We include hyperparameters of our private linear probing results on ImageNet in Tab. 23. In Tab. 23, we use  $\sigma_{1}$  for the private mean estimation step and  $\sigma_{2}$  for DP-SGD. Because we are using the full batch, we can use the composition theorem in Gaussian differential privacy [17] (Theorem 2.7 and Corollary 3.3) to compute the privacy loss. With  $T$  steps in DP-SGD, our private linear probing is equivalent to a one-step Gaussian mechanism with noise multiplier  $\sigma$ , where
$$
\sigma = \frac {1}{\sqrt {\frac {1}{\sigma_ {1} ^ {2}} + \frac {T}{\sigma_ {2} ^ {2}}}}
$$
We can then compute  $(\varepsilon, \delta)$ -DP by computing the privacy curve of Gaussian mechanism [3, 17].
According to Panda et al. [54], as  $\varepsilon$  increases, the total step size will also increase to achieve better performance. Therefore in Tab. 23, we use  $T = 100$  for  $\varepsilon = 1$  and  $T = 200$  for  $\varepsilon = 8$ . In addition to Tab. 23, we also provide a few more observations during the hyperparameter search.  $T = 100$  for  $\varepsilon = 8$  also achieves compelling result that is close to  $39\%$ . When we fix  $\varepsilon = 8$ , as we increase  $T$ , the performance improves. Further increasing  $T$  would continue to improve the performance. As the number of steps  $T$  increases, the corresponding optimal learning rate would decrease. This is consistent with previous work [54].
We note that Sander et al. [57] also tried training the ViT model on ImageNet but concluded that it does not perform as well as ResNet. Our explanation for this is that ViT requires pretraining data because the architecture does not encode any natural prior, whereas CNNs naturally have a prior. The nature of convolutional filters biases CNNs to extract features with spatial locality. As we observe in Sec. 2, the impact of pretraining data is mostly at the initialization by giving the model a prior, so it stands to reason that the missing piece in utilizing ViT for DP training on ImageNet is learning a random prior from Shaders-21k [6].
Table 23: Hyperparameters for linear probing on ImageNet.  $\sigma_{1}$  is for private mean estimation and  $\sigma_{2}$  is for DP-SGD. Other hyperparameters include batch size = full, SGD optimizer with momentum = 0.9. For the linear layer, bias = False and zero initialization. We do not employ any additional regularization or learning rate schedule.
ε	Steps T	σ1	σ2	learning rate
ε = 1	100	71	43	3
ε = 8	200	14	9.33	10
H Two-stage training with private data
After pretraining with synthetic data, DP-RandP first training the classifier head (Phase II) and then tuning all hyperparameter (Phase III). Tab. 24 summarizes the results in the main paper and presents the comparison of our full DP-RandP, DP-RandP without Phase II and the baseline [15]. Note that DP-RandP without Phase III is not included in Tab. 24 as training the linear layer only has diminishing returns: the non-private baseline of DP-RandP w/o Phase III is  $74.05\%$ , which is worse than DP-RandP w/o Phase II at  $\varepsilon = 2$ . Tab. 24 shows the importance of combining both Phase II and Phase III in DP-RandP.
Table 24: The importance of phases in DP-RandP. Evaluation of test accuracy (%) on CIFAR10.
Phases	ε = 1	ε = 2	ε = 3	ε = 4	ε = 6	ε = 8
DP-RandP	72.32	77.25	79.99	81.88	84.01	85.26
DP-RandP w/o Phase II	69.03	75.31	78.44	80.56	82.96	84.45
Phase III only (De et al. [15])	56.8	64.9	69.2	71.9	71.0	79.5
Our DP-RandP uses synthetic data for pretraining to give a warm initialization for two-stage training with private data. Tab. 25 shows the two-stage training with private data with random initialization. As noted in Tab. 13, we use  $\sigma = 9.3$  for  $\varepsilon = 1$  while De et al. [15] use  $\varepsilon = 10$  for the same number of 875 steps (equals to 73 epochs). This is because we only round  $\sigma$  up to 0.1 and we ensure the  $\sigma$  we use in Tab. 13 will not exceed the designed privacy bound. As we add less noise compared to De et al. [15], the baseline result, i.e., directly updating the full parameters during training (0 epoch for Phase II), is 57.46.
Table 25: Test accuracy (%) of two-stage training with private data by random initialization on CIFAR10.
Epochs for Phase II	0	1	2	4	8	12	16	20	50	73
Accuracy (%)	57.46	57.01	56.27	55.04	54.45	54.09	53.15	52.26	47.00	25.80
Std.	0.48	0.69	0.57	0.90	0.80	0.41	0.91	0.97	1.53	2.08
Tab. 25 shows that when the feature extractor is randomly initialized, if we train the head classifier with more steps while keeping the same total steps, the performance will decrease compared to directly finetuning the whole network. Our analysis of this is that, the feature extractor is not encoded with any prior and it is better to update the full network in the whole training procedure to learn more information.