SlowGuess/NeurIPS_2023_OCR huggingface dataset:
based on [4, Thm. 21]. This can be shown to be bounded above by  $\frac{1}{\sigma}\sqrt{2\log\frac{1}{\delta}} +\frac{1}{2\sigma^2}$  [44, Prop. 3]. By Theorem 3, it follows that the operation  $\mathcal{A}(\cdot)$  is also  $(\varepsilon_{\delta},\delta)$ -LiDP.
Auditing LiDP. We follow the recipe of Algorithm 1. We set the rejection region  $\pmb{R} = R_{\tau}(\pmb{c}_{K})$  as a function of the canary  $c_{K}$  that differs between  $D_0$  and  $D_{1}$ , where
$$
R _ {\tau} \left(\boldsymbol {c} _ {j}\right) := \left\{u \in \mathbb {R} ^ {d}: \langle u, \boldsymbol {c} _ {j} \rangle \geq \tau \right\}, \tag {26}
$$
and  $\tau \in \mathbb{R}$  is a tuned threshold.
Figure 9: Effect of the number  $k$  of canaries on the empirical lower bound  $\hat{\varepsilon}$  from auditing the Gaussian mechanism for DP and LiDP. The shaded are denotes the standard error over 25 random seeds.
We evaluate empirical privacy auditing methods by how large the lower bound  $\hat{\varepsilon}$  is — the higher the lower bound, the better the confidence interval.
Methods Compared. An empirical privacy auditing method is defined by the type of privacy auditing (DP or LiDP) and the type of confidence intervals. We compare the following auditing methods:
- DP + Wilson: We audit the usual  $(\varepsilon, \delta)$ -DP with  $K = 1$  canary. This corresponds exactly to auditing LiDP with  $K = 1$ . We use the 1st-Order Wilson confidence intervals for a fair comparison with the other LiDP auditing methods. This performs quite similarly to the other intervals used in the literature, cf. Figure 7.
- LiDP + 1st-Order Wilson: We audit LiDP with  $K$  canaries with the 1st-Order Wilson confidence interval. This method cannot leverage the shrinking of the confidence intervals from higher-order estimates.
- LiDP + 2nd/4th-Order Wilson: We audit LiDP with  $K > 1$  canaries using the higher-order Wilson confidence intervals. Unless stated otherwise, we use  $m = K$  test canaries.
Parameters of the Experiment. We vary the following parameters in the experiment:
Number of trials  $n\in \{2^8,2^{10},\dots ,2^{16}\} .$
- Number of canaries  $k \in  \{ 1,2,{2}^{2},\ldots ,{2}^{10}\}$  .
- Dimension  $d \in \{10^2, 10^3, \ldots, 10^6\}$ .
Figure 10: Effect of the data dimension  $d$  on the empirical lower bound  $\hat{\varepsilon}$  from auditing the Gaussian mechanism for DP and LiDP. The shaded area denotes the standard error over 25 random seeds.
DP upper bound  $\varepsilon \in \{1,2,4,8\}$
We fix the DP parameter  $\delta = 10^{-5}$  and the failure probability  $\beta = 0.05$ .
Tuning the threshold  $\tau$ . For each confidence interval scheme, we repeat the estimation of the lower bound  $\hat{\varepsilon}(\tau)$  for a grid of thresholds  $\tau \in \Gamma$  on a holdout set of  $n$  trials. We fix the best threshold  $\tau^{*} = \arg \max_{\tau \in \Gamma} \hat{\varepsilon}(\tau)$  that gives the largest lower bound  $\hat{\varepsilon}(\tau)$  from the grid  $\Gamma$ . We then fix the threshold  $\tau^{*}$  and report numbers over a fresh set of  $n$  trials.
Randomness and Repetitions. We repeat each experiment 25 times (after fixing the threshold) with different random seeds and report the mean and standard error.
E.2 Additional Experimental Results
Effect of the Number  $m$  of Test Canaries. In Figure 13, we vary the number  $m$  of test canaries while keeping the number  $K$  of training canaries fixed. We see that the 2nd and 4th-order Wilson estimators lead to monotonically improving lower bound  $\hat{\varepsilon}$  as  $m$  increases. However, these improvements soon flatten out between  $m = 8$  and  $m = 32$  depending on other parameters. In particular, the default value of  $m = K$  is nearly or equal to the best value of  $m$  for  $K \geq 16$ .
We give additional experimental results, expanding on the plots shown in Figures 3 and 4:
- Figure 8 shows the effect of varying the number of trials  $n$ , similar to Figure 3 (left).
Figure 11: Effect of the number  $k$  of canaries on the moment estimates employed by the higher-order Wilson intervals in auditing LiDP.
- Figure 9 shows the effect of varying the number of canaries  $k$ , similar to Figure 3 (middle).
- Figure 10 shows the effect of varying the data dimension  $d$ , similar to Figure 3 (right).
- Figure 11 shows the effect of varying the number of canaries  $k$  on the moment estimates, similar to Figure 4 (right).
- Figure 12 shows the effect of varying the data dimension  $d$  on the moment estimates, similar to Figure 4 (right).
We observe that the insights discussed in §4 hold across a wide range of the parameter values. In addition, we make the following observations.
The benefit of higher-order confidence estimators. We see from Figures 8 to 10 that the higher-order Wilson estimators lead to larger relative improvements at smaller  $\varepsilon$ . On the other hand, they perform similarly at large  $\varepsilon$  (e.g.,  $\varepsilon = 8$ ) to the lower-order estimators.
4th-Order Wilson vs. 2nd-Order Wilson. We note that the 4th-order Wilson interval outperforms the 2nd-order Wilson interval at  $\varepsilon = 1$ , while the opposite is true at large  $\varepsilon = 8$ . At intermediate values of  $\varepsilon$ , both behave very similarly. We suggest the 2nd-order Wilson interval as a default because it is nearly as good as or better than the 4th-order variant across the board, but is easier to implement.
Figure 12: Effect of the data dimension on the moment estimates employed by the higher-order Wilson intervals in auditing LiDP.
Figure 13: Effect of varying the number  $m$  of test canaries with the number  $K$  of canaries inserted in the training being fixed.
F Experiments: Details and More Results
We describe the detailed experimental setup here.
F.1 Training Details: Datasets, Models
We consider two datasets, FMNIST and Purchase-100. Both are multiclass classification datasets trained with the cross-entropy loss using stochastic gradient descent (without momentum) for a fixed epoch budget.
- FMNIST: FMNIST or FashionMNIST [65] is a classification of  $28 \times 28$  grayscale images of various articles of clothing into 10 classes. It contains 60K train images and 10K test images. The dataset is available under the MIT license. We experiment with two models: a linear model and a multi-layer perceptron (MLP) with 2 hidden layers of dimension 256 each. We train each model for 30 epochs with a batch size of 100 and a fixed learning rate of 0.02 for the linear model and 0.01 for the MLP.
- Purchase-100: The Purchase dataset is based on Kaggle's "acquire valued shoppers" challenge [19] that records the shopping history of 200K customers. The dataset is available publicly on Kaggle but the owners have not created a license as far as we could tell. We use the preprocessed version of  $[53]^3$  where the input is a 600-dimensional binary vector encoding the shopping history. The classes are obtained by grouping the records into 100 clusters using  $k$ -means. We use a fixed subsample of 20K training points and 5K test points. The model is a MLP with 2 hidden layers of 256 units each. It is trained for 100 epochs with a batch size of 100 and a fixed learning rate of 0.05.
F.2 DP and Auditing Setup
We train each dataset-model pair with DP-SGD [1]. The noise level is calibrated so that the entire training algorithm satisfies  $(\varepsilon, \delta)$ -differential privacy with  $\varepsilon$  varying and  $\delta = 10^{-5}$  fixed across all experiments. We tune the per-example gradient clip norm for each dataset, model, and DP parameter  $\varepsilon$  so as to maximize the validation accuracy.
Auditing Setup. We follow the LiDP auditing setup described in Appendix B.2. Recall that auditing LiDP with  $K = 1$  canaries and corresponds exactly with auditing DP. We train  $n \in \{125, 250, 500, 1000\}$  trials, where each trial corresponds to a model trained with  $K$  canaries in each run. We try two methods for canary design (as well as their corresponding rejection regions), as discussed in Appendix D: data poisoning and random gradients.
With data poisoning for FMNIST, we define the canary distribution  $\mathcal{P}_{\mathrm{canary}}$  as the uniform distribution over the last  $p = 284$  principal components (i.e., principal components 500 to 784). For Purchase-100, we use the uniform distribution over the last  $p = 300$  principal components (i.e., principal components 300 to 600).
For both settings, we audit only the final model, assuming that we do not have access to the intermediate models. This corresponds to the blackbox auditing setting for data poisoning and a graybox setting for random gradient canaries.
F.3 Miscellaneous Details
Hardware. We run each job on an internal compute cluster using only CPUs (i.e., no hardware accelerators such as GPUs were used). Each job was run with 8 CPU cores and 16G memory.
F.4 Additional Experimental Results
F.4.1 Comparison to Bayesian Auditing of DP
We compare the proposed LiDP auditing approach with the Bayesian approach of Zanella-Béguelin et al. [69]. This approach constructs a posterior distribution over the true and false positive rates by updating the non-informative Jeffreys priors using empirical observations. They use Bayesian credible intervals (as opposed to confidence intervals in our case) computed using numerical quadrature.
We compare the proposed LiDP auditing with  $K = m > 1$  with the Bayesian approach of [69] with  $K = m = 1$  in Figure 14 using the code open-sourced by the authors. Owing to the increased runtime in the numerical estimation of the  $\varepsilon$  of the Bayesian approach, we tune its threshold  $\tau$  to separate the null and alternate hypotheses (cf. line 5 of Algorithm 1) for the Clopper-Pearson interval.
We observe that the LiDP auditing approach generally provides improved estimates when compared to this baseline. Note that the reason behind the improvement for the Bayesian approach of [69] is by capture the joint distribution of the true and false positive rates, while the improvement behind LiDP
<sup>3</sup>https://github.com/privacytrustlab/datasets
<sup>4</sup>https://github.com/microsoft/responsible-ai-toolbox-privacy
Figure 14: Comparing the proposed LiDP auditing approach to the Bayesian credible interval approach of Zanella-Béguelin et al. [69].
Figure 15: Comparison of higher-order Bernstein and Wilson confidence intervals for LiDP auditing.
auditing is by a clever use of randomized tests. Both sources of improvements are orthogonal to each other. This suggests an interesting future research direction: adapt Bayesian credible intervals for LiDP auditing to get the best of both worlds.
F.4.2 Comparison of Asymptotic vs. Non-asymptotic Confidence Intervals
We compare the asymptotic Wilson intervals (used primarily in the experiments) to the non-asymptotic Bernstein intervals (used primarily for the theory) in Figure 15. The results show that the Wilson intervals give larger empirical lower bounds in general with all other parameters remaining the same. This is as expected because the Wilson intervals are an asymptotic tightening of the Bernstein intervals as shown in Figure 2.
F.4.3 Experiments Across Different Parameter Settings
We give the following plots to augment Table 1 and Figure 6:
- Figure 16: plot of the test accuracy upon adding  $K$  canaries to the training process.
- Figure 17: experimental results for FMNIST linear model.
- Figure 18: experimental results for FMNIST MLP model.
- Figure 19: experimental results for Purchase MLP model.
Figure 16: Test accuracy versus the number of canaries  $K$ . We plot the mean over 1000 training runs (the standard error in under  $10^{-5}$ ). Adding multiple canaries to audit LiDP does not have any impact on the final test accuracy of the model trained with DP.
Linear Model on FMNIST with Data Poisoning Canary
Linear Model on FMNIST with Random Gradient Canary
Figure 17: Experimental results for FMNIST linear model (top two: varying  $n$ , bottom two: varying  $K$ ).
Figure 18: Experimental results for FMNIST MLP model (top two: varying  $n$ , bottom two: varying  $K$ ).
MLP Model on Purchase-100 with Data Poisoning Canary
MLP Model on Purchase-100 with Random Gradient Canary
Figure 19: Experimental results for Purchase-100 MLP model (top two: varying  $n$ , bottom two: varying  $K$ ).