SlowGuess/NeurIPS_2023_OCR huggingface dataset:
1: Privacy Parameters for DP-SGD:  $k$  - number of mechanisms to be composed,  $\sigma$  - noise multiplier,  $q$  - subsampling rate,  $C$  - clipping ratio.
2:
3: // Step 1: Estimate Privacy Parameter.
4: Obtain a privacy parameter estimate  $(\varepsilon, \delta^{\mathrm{est}})$  from a DP accountant (e.g., FFT/GDP/MC accountant). Set the smoothing factor  $\tau = 0.99$  (one can also adjust the value of  $\tau$  according to the privacy guarantee and runtime requirements).
5:
6: // Step 2: Verify Privacy Parameter.
7:
8: // Step 2.1: Derive the required amount of samples for privacy verification.
9: Compute the number of required samples  $m$  according to Theorem 13, where the second moment upper bound  $\nu$  is computed according to Corollary 22 for Simple MC estimator, or Theorem 23 for Importance Sampling estimator. Set  $\rho = (1 + \tau) / 2$  and set  $\Delta$  according to Theorem 14 for utility guarantee.
10:
11: // Step 2.2: Estimate privacy parameter with MC estimator.
12: If use Simple MC estimator: Obtain i.i.d. samples  $\{t_i\}_{i=1}^m$  from  $P$ . Compute simple MC estimate  $\widehat{\delta} = \frac{1}{m}\sum_{i=1}^{m}(1 - e^{\varepsilon - y_i})_+$  with PRV samples  $y_i = \log\left(\frac{P(t_i)}{Q(t_i)}\right)$ ,  $i = 1\dots m$ .
13: If use Importance Sampling estimator: compute MC estimate according to Definition 18.
14:
15: // Step 3: Release according to verification result.
16: if  $\widehat{\delta} < \frac{\delta^{\mathrm{est}}}{\tau} - \Delta$  then release the result of DP-SGD.
17: else terminate program.
It can also be extended to the heterogeneous case easily.
H Experiment Settings & Additional Results
H.1 GPU Acceleration for MC Sampling
MC-based techniques are well-suited for parallel computing and GPU acceleration due to their nature of repeated sampling. One can easily utilize PyTorch's CUDA functionality, e.g.,
$$
t o r c h. r a n d n (s i z e = (k, m)). c u d a () * s i g m a + m u
$$
, to significantly boost the computational efficiency. Figure 9 (a) shows that when using a NVIDIA A100-SXM4-80GB GPU, the execution time of sampling Gaussian mixture  $((1 - q)\mathcal{N}(0,\sigma^2) + q\mathcal{N}(1,\sigma^2))$  can be improved by  $10^{3}$  times compared with CPU-only scenario. Figure 9 (b) shows the predicted runtime for different target false positive rates for  $k = 1000$ . We vary  $\sigma$  and set the target false positive rate as the smallest  $s\times 10^{-r}$  that is greater than  $\delta_Y(\varepsilon)$ , where  $s\in \{1,5\}$  and  $r$  is positive integer. We set  $\delta^{\mathrm{est}} = 0.8\delta_{Y}(\varepsilon)$ , and  $\tau ,\rho ,\Delta$  are set as the heuristics introduced in the previous sections. The runtime is predicted by the number of required samples for the given false positive rate. As we can see, even when we target at  $10^{-10}$  false positive rate (which means that  $\Delta \approx 10^{-10}$ ), the clock time is still acceptable (around 3 hours).
Figure 9: The execution time of sampling Gaussian mixture  $((1 - q)\mathcal{N}(0,\sigma^2) + q\mathcal{N}(1,\sigma^2))$  when only using CPU and when using a NVIDIA A100-SXM4-80GB GPU.
H.2 Experiment for Evaluating EVR Paradigm
H.2.1 Settings
For Figure 1 & Figure 3, the FFT-based method has hyperparameter being set as  $\varepsilon_{\mathrm{error}} = 10^{-3}$ ,  $\delta_{\mathrm{error}} = 10^{-10}$ . For the GDP-Edgeworth accountant, we use the second-order expansion and uniform bound, following [41].
For Figure 4 (as well as Figure 11), the BEiT [4] is first self-supervised pretrained on ImageNet-1k and then trained finetuned on ImageNet-21k, following the state-of-the-art approach in [34]. For DP-GD training, we set  $\sigma$  as 28.914, clipping norm as 1, learning rate as 2, and we train for at most 60 iterations, and we only finetune the last layer on CIFAR-100.
H.2.2 Additional Results
$k \to \varepsilon_{Y^{(k)}}(\delta)$  curve. We show additional results for a more common setting in privacy-preserving machine learning where one set a target  $\delta$  and try to find  $\varepsilon_{Y^{(k)}}(\delta)$  for different  $k$ , the number of individual mechanisms in the composition. We use  $Y^{(k)}$  to stress that the PRV  $Y$  is for the composition of  $k$  mechanisms. Such a scenario can happen when one wants to find the optimal stopping iteration for training a differentially private neural network.
Figure 10 shows such a result for Poisson Subsampled Gaussian where we set subsampling rate 0.01,  $\sigma = 2$ , and  $\delta = 10^{-5}$ . We set  $\varepsilon_{\mathrm{error}} = 10^{-1}$ ,  $\delta_{\mathrm{error}} = 10^{-10}$  for the FFT method. The estimate in this case is obtained by fixing  $\delta^{\mathrm{est}} = \tau \delta$  and find the corresponding estimate for  $\varepsilon$  through FFT-based method [19]. As we can see, the EVR paradigm achieves a much tighter privacy analysis compared with the upper bound derived by FFT-based method. The runtime of privacy verification in this case is  $< 15$  minutes for all  $k$ s.
Figure 10:  $k \rightarrow \varepsilon_{Y^{(k)}}(\delta)$  curve for Poisson Subsampled Gaussian mechanism for subsampling rate 0.01,  $\sigma = 2$ , and  $\delta = 10^{-5}$ . When running on an NVIDIA A100-SXM4-80GB GPU, the runtime of privacy verification is  $< 15$  minutes.
Privacy-Utility Tradeoff. We show additional results for the privacy-utility tradeoff curve when finetuning ImageNet-pretrained BEiT on CIFAR100 dataset with DP stochastic gradient descent (DP-SGD). For DP-SGD training, we set  $\sigma$  as 5.971, clipping norm as 1, learning rate as 0.2, momentum as 0.9, batch size as 4096, and we train for at most 360 iterations (30 epochs). We only finetune the last layer on CIFAR-100.
As shown in Figure 11 (a), the EVR paradigm provides a better utility-privacy tradeoff compare with the traditional upper bound method. In Figure 11 (b), we show the runtime of DP verification when  $\rho = (1 + \tau) / 2$  and we set  $\Delta$  according to Theorem 14 (which ensures EVR's failure probability is negligible). The runtime is estimated on an NVIDIA A100-SXM4-80GB GPU. As we can see, it only takes a few minutes for privacy verification, which is short compared with hours of model training.
Figure 11: (a) Utility-privacy tradeoff curve for fine-tuning ImageNet-pretrained BEiT [4] on CIFAR100 when  $\delta = 10^{-12}$  with DP-SGD. We follow the training hyperparameters from [34]. (b) Runtime of privacy verification in the EVR paradigm. For fair comparison, we set  $\rho = (1 + \tau) / 2$  and set  $\Delta$  according to Theorem 14, which ensures EVR's failure probability is around  $O(\delta)$ . For (b), the runtime is estimated on an NVIDIA A100-SXM4-80GB GPU.
H.3 Experiment for Evaluating MC Accountant
H.3.1 Settings
Evaluation Protocol. We use  $Y^{(k)}$  to stress that the PRV  $Y$  is for the composition of  $k$  Poisson Subsampled Gaussian mechanisms. For the offline setting, we make the following two kinds of plots: (1) the relative error in approximating  $\varepsilon \mapsto \delta_{Y^{(k)}}(\varepsilon)$  (for fixed  $k$ ), and (2) the relative error in  $k \mapsto \varepsilon_{Y^{(k)}}(\delta)$  (for fixed  $\delta$ ), where  $\varepsilon_{Y^{(k)}}(\delta)$  is the inverse of  $\delta_{Y^{(k)}}(\varepsilon)$  from (2). For the online setting, we make the following two kinds of plots: (1) the relative error in approximating  $k \mapsto \varepsilon_{Y^{(k)}}(\delta)$  (for fixed  $\delta$ ), and (2)  $k \mapsto$  cumulative time for privacy accounting until  $k$ th iteration.
MC Accountant. We use the importance sampling technique with the tilting parameter being set according to the heuristic described in Definition 18.
Baselines. We compare MC accountant against the following state-of-the-art DP accountants with the following settings:
- The state-of-the-art FFT-based approach [19]. The setting of  $\varepsilon_{\mathrm{error}}$  and  $\delta_{\mathrm{error}}$  is specified in the next section.
- CLT-based GDP accountant [7].
- GDP-Edgeworth accountant with second-order expansion and uniform bound.
- The Analytical Fourier Accountant based on characteristic function [44], with double quadrature approximation as this is the practical method recommended in the original paper.
H.3.2 Additional Results for Online Accounting
Figure 12 and 13 show the online accounting results for  $(\sigma, \delta, q) = (0.5, 10^{-5}, 10^{-3})$  and  $(\sigma, \delta, q) = (0.5, 10^{-13}, 10^{-3})$ , respectively. For the setting of  $(\sigma, \delta, q) = (0.5, 10^{-5}, 10^{-3})$ , we can see that the MC accountant achieves a comparable performance with a shorter runtime. For the setting of  $(\sigma, \delta, q) = (0.5, 10^{-13}, 10^{-3})$ , we can see that the MC accountant achieves significantly better performance compared to the state-of-the-art FFT accountant (and again, with a shorter runtime). This showcases the MC accountant's efficiency and accuracy in online setting.
Figure 12: Experiment for Composing Subsampled Gaussian Mechanisms in the Online Setting with hyperparameter  $(\sigma, \delta, q) = (0.5, 10^{-5}, 10^{-3})$ . (a) Compares the relative error in approximating  $k \mapsto \varepsilon_{Y}(\delta)$ . The error bar for MC accountant is the variance taken over 5 independent runs. Note that the y-axis is in the log scale. (b) Compares the cumulative runtime for online privacy accounting. We did not show AFA [44] as it does not terminate in 24 hours.
Figure 13: Experiment for Composing Subsampled Gaussian Mechanisms in the Online Setting with hyperparameter  $(\sigma, \delta, q) = (0.5, 10^{-13}, 10^{-3})$ . (a) Compares the relative error in approximating  $k \mapsto \varepsilon_{Y}(\delta)$ . The error bar for MC accountant is the variance taken over 5 independent runs. Note that the y-axis is in the log scale. (b) Compares the cumulative runtime for online privacy accounting. We did not show AFA [44] as it does not terminate in 24 hours.
H.3.3 Additional Results for Offline Accounting
In this experiment, we set the number of samples for MC accountant as  $10^{7}$ , and the parameter for FFT-based method as  $\varepsilon_{\mathrm{error}} = 10^{-3}$ ,  $\delta_{\mathrm{error}} = 10^{-10}$ . The parameters are controlled so that the MC accountant is faster than FFT-based method, as shown in Table 1. Figure 14 (a) shows the offline accounting results for  $\varepsilon \mapsto \delta_{Y(k)}(\varepsilon)$  when we set  $(\sigma, q, k) = (0.5, 10^{-3}, 1000)$ . As we can see, the performance of MC accountant is comparable with the state-of-the-art FFT method. In Figure 15 (a), we decreases  $q$  to  $10^{-5}$ . Compared against baselines, MC approximations are significantly more accurate for larger  $\varepsilon$ , compared with the FFT accountant. Figure 14 (b) shows the offline accounting results for  $k \mapsto \varepsilon_{Y}(\delta)$  when we set  $(\sigma, q, \delta) = (0.5, 10^{-3}, 10^{-5})$ . Similarly, MC accountant performs comparably as FFT accountant. However, when we decrease  $q$  to  $10^{-5}$  and  $\delta$  to  $10^{-14}$  (Figure 15 (b)), MC accountant significantly outperforms FFT accountant. This illustrates that MC accountant performs well in all regimes, and is especially more favorable when the true value of  $\delta_{Y}(\varepsilon)$  is tiny.
Figure 14: Experiment for Composing Subsampled Gaussian Mechanisms: (a) Compares the relative error in approximating  $\varepsilon \mapsto \delta_{Y^{(k)}}(\varepsilon)$  where we set  $\sigma = 0.5, k = 1000, q = 10^{-3}$ . (b) Compares the relative error in  $k \mapsto \varepsilon_{Y}(\delta)$  where we set  $\sigma = 0.5, \delta = 10^{-5}, q = 10^{-3}$ . The error bar for MC accountant is the variance taken over 5 independent runs. Note that the y-axis is in the log scale.
Table 1: Runtime for  $k = {1000}$  . AFA GDP GDP-E FFT MC-IS  ${18.63} \times  {10}^{-4}$  1.50 3.01 2.31
Figure 15: Experiment for Composing Subsampled Gaussian Mechanisms: (a) Compares the relative error in approximating  $\varepsilon \mapsto \delta_{Y(k)}(\varepsilon)$  where we set  $\sigma = 0.5$ ,  $k = 100$ ,  $q = 10^{-5}$ . (b) Compares the relative error in  $k \mapsto \varepsilon_{Y}(\delta)$  where we set  $\sigma = 0.5$ ,  $\delta = 10^{-14}$ ,  $q = 10^{-5}$ . The error bar for MC accountant is the variance taken over 5 independent runs. Note that the y-axis is in the log scale. The curves for AFA, GDP and GDP-Edgeworth are overlapped with each other.