SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Proposition 9. Consider Algorithm 3 with inputs  $n$  i.i.d. samples  $\pmb{x}^{(1)},\dots,\pmb{x}^{(n)} \stackrel{i.i.d.}{\sim} XBern_K(\mu_1,\dots,\mu_K)$  for some  $K \geq 2$ . Then, its outputs  $\underline{\mu}_1,\overline{\mu}_1$  satisfy  $\mathbb{P}(\mu_1 \geq \underline{\mu}_1) \geq 1 - \beta$  and  $\mathbb{P}(\mu_1 \leq \overline{\mu}_1) \geq 1 - \beta$  and  $\mathbb{P}(\underline{\mu}_1 \leq \mu_1 \leq \overline{\mu}_1) \leq 1 - \frac{3\beta}{2}$ .
Proof. Algorithm 3 computes the correct 2nd moment  $m_2^{(i)}$  due to Proposition 6. Applying Bernstein's inequality to  $m_2$ , we get  $\mathbb{P}(\mu_2 \leq \overline{\mu}_2) \geq 1 - \beta / 2$  (see also the proof of Proposition 8). Next, from Bernstein's inequality applied to  $m_1$ , we leverage (17) to say that with probability at least  $1 - \beta / 2$ , we have
$$
\mu_ {1} - \hat {\mu} _ {1} \leq \frac {2}{3 n} \log \frac {2}{\beta} + \sqrt {\frac {2}{n} \log \frac {2}{\beta} \left(\frac {\mu_ {1}}{K} - \mu_ {1} ^ {2} + \frac {K - 1}{K} \mu_ {2}\right)}.
$$
Together with the result on  $\overline{\mu}_2$ , we have with probability at least  $1 - \beta$  that
$$
\mu_ {1} - \hat {\boldsymbol {\mu}} _ {1} \leq \frac {2}{3 n} \log \frac {2}{\beta} + \sqrt {\frac {2}{n} \log \frac {2}{\beta} \left(\frac {\mu_ {1}}{K} - \mu_ {1} ^ {2} + \frac {K - 1}{K} \overline {{\boldsymbol {\mu}}} _ {2}\right)}.
$$
Algorithm 4 Fourth-Order Bernstein Intervals
1: For each  $i \in [n]$ , set  $\pmb{m}_1^{(i)} = (1 / K)\sum_{j=1}^K \pmb{x}_j^{(i)}$  and for  $\ell = 1,2,3$ :  $\pmb{m}_{\ell+1}^{(i)} = \pmb{m}_{\ell}^{(i)}\left(\frac{K\pmb{m}_1^{(i)} - \ell}{K - \ell}\right)$ .
2: Set  $\hat{\pmb{\mu}}_{\ell} = (1 / n)\sum_{i = 1}^{n}\pmb{m}_{\ell}^{(i)}$  for  $\ell = 1,2,3,4$
3: For  $\ell = 3,4$ , set  $\overline{\pmb{\mu}}_{\ell}$  as the unique solution of  $x\in [\hat{\pmb{\mu}}_{\ell},1]$  such that
$$
x - \hat {\mu} _ {\ell} - \sqrt {\left(\frac {2}{n} \log \frac {4}{\beta}\right) x (1 - x)} = \frac {2}{3 n} \log \frac {4}{\beta}.
$$
if it exists, else set  $\overline{\mu}_{\ell} = 1$
4: Set  $\overline{\mu}_2$  as the unique solution, if it exists, of  $x\in [\hat{\mu}_2,1]$  such that
$$
x - \hat {\mu} _ {2} - \sqrt {\left(\frac {2}{n} \log \frac {4}{\beta}\right) \sigma_ {2} ^ {2} (x , \overline {{\mu}} _ {3} , \overline {{\mu}} _ {4})} = \frac {2}{3 n} \log \frac {4}{\beta}
$$
where  $\sigma_2^2 (\cdot ,\cdot ,\cdot)$  is as defined in (18). Else set  $\overline{\mu}_2 = 1$
5: Set  $\underline{\boldsymbol{\mu}}_1$  as the unique solution of  $x\in [0,\hat{\boldsymbol{\mu}}_1]$  such that
$$
\hat {\mu} _ {1} - x - \sqrt {\left(\frac {2}{n} \log \frac {4}{\beta}\right) \left(\frac {x}{k} - x ^ {2} + \frac {K - 1}{K} \bar {\mu} _ {2}\right)} = \frac {2}{3 n} \log \frac {4}{\beta}
$$
6: Set  $\overline{\pmb{\mu}}_1$  as the unique solution of  $x\in [\hat{\pmb{\mu}}_1,1]$  such that
$$
x - \hat {\mu} _ {1} - \sqrt {\left(\frac {2}{n} \log \frac {4}{\beta}\right) \left(\frac {x}{k} - x ^ {2} + \frac {K - 1}{K} \overline {{\mu}} _ {2}\right)} = \frac {2}{3 n} \log \frac {4}{\beta}
$$
7: return  $\underline{\mu}_1, \overline{\mu}_1$ .
We can verify that the output  $\overline{\mu}_1$  is the largest value of  $\mu_1 \leq 1$  that satisfies the above inequality. Similarly,  $\underline{\mu}_1$  is obtained as a lower Bernstein bound on  $m_1$  with probability at least  $1 - \beta$ . By the union bound,  $\underline{\mu}_1 \leq \mu_1 \leq \overline{\mu}_1$  holds with probability at least  $1 - 3\beta / 2$ , since we have three invocations of Bernstein's inequality, each with a failure probability of  $\beta / 2$ .
C.1.3 Fourth-Order Bernstein Intervals
The fourth-order Bernstein interval depends on the first four empirical moments  $\mu_1, \ldots, \mu_4$ . It is given in Algorithm 4. The derivation of the interval is based on the calculation
$$
\begin{array}{l} \operatorname {V a r} \left(\boldsymbol {m} _ {2}\right) = \mathbb {E} \left[ \boldsymbol {m} _ {2} ^ {2} \right] - \mu_ {2} ^ {2} \\ = \frac {2 \mu_ {2}}{K (K - 1)} - \mu_ {2} ^ {2} + \frac {4 (K - 2)}{K (K - 1)} \mu_ {3} + \frac {(K - 2) (K - 3)}{K (K - 1)} \mu_ {4} \\ = \frac {2 \mu_ {2} (1 - \mu_ {2})}{K (K - 1)} + \frac {4 (K - 2)}{K (K - 1)} \left(\mu_ {3} - \mu_ {2} ^ {2}\right) + \frac {(K - 2) (K - 3)}{K (K - 1)} \left(\mu_ {4} - \mu_ {2} ^ {2}\right) \tag {18} \\ =: \sigma_ {2} ^ {2} (\mu_ {2}, \mu_ {3}, \mu_ {4}). \\ \end{array}
$$
Proposition 10. Consider Algorithm 4 with inputs  $n$  i.i.d. samples  $\pmb{x}^{(1)},\dots,\pmb{x}^{(n)} \stackrel{i.i.d.}{\sim} XBern_K(\mu_1,\dots,\mu_K)$  for some  $K \geq 4$ . Then, its outputs  $\underline{\mu}_1,\overline{\mu}_1$  satisfy  $\mathbb{P}(\mu_1 \geq \underline{\mu}_1) \geq 1 - \beta$  and  $\mathbb{P}(\mu_1 \leq \overline{\mu}_1) \geq 1 - \beta$  and  $\mathbb{P}(\underline{\mu}_1 \leq \mu_1 \leq \overline{\mu}_1) \leq 1 - \frac{5\beta}{4}$ .
Algorithm 5 First-Order Wilson Intervals
Output: Asymptotic confidence intervals  $[\underline{\mu}_1, \overline{\mu}_1]$  such that  $\lim_{n \to \infty} \mathbb{P}(\mu_1 < \underline{\mu}_1) \leq \beta$  and  $\lim_{n \to \infty} \mathbb{P}(\mu_1 > \overline{\mu}_1) \leq \beta$ .
1: Set  $\underline{\mu}_1 < \overline{\mu}_1$  as the roots of the quadratic in  $x$ :
$$
\left(n + Z _ {\beta} ^ {2}\right) x ^ {2} - \left(2 n \hat {\mu} _ {1} + Z _ {\beta} ^ {2}\right) x + n \hat {\mu} _ {1} ^ {2} = 0.
$$
2: return  $\underline{\mu}_1, \overline{\mu}_1$ .
Algorithm 6 Second-Order Wilson Intervals
1: For each  $i \in [n]$ , set  $\pmb{m}_1^{(i)} = (1 / K)\sum_{j=1}^{K}\pmb{x}_j^{(i)}$  and  $\pmb{m}_2^{(i)} = \pmb{m}_1^{(i)}\left(\frac{K\pmb{m}_1^{(i)} - 1}{K - 1}\right)$ .
2: Set  $\hat{\pmb{\mu}}_{\ell} = (1 / n)\sum_{i = 1}^{n}\pmb{m}_{\ell}^{(i)}$  for  $\ell = 1,2$
3: Let  $\overline{\mu}_2$  be the larger root of the quadratic in  $x$ :
$$
\left(n + Z _ {\beta / 2} ^ {2}\right) x ^ {2} - \left(2 n \hat {\mu} _ {2} + Z _ {\beta / 2} ^ {2}\right) x + n \hat {\mu} _ {2} ^ {2} = 0.
$$
4: Set  $\underline{\mu}_1 < \overline{\mu}_1$  as the roots of the quadratic in  $x$ :
$$
\left(n + Z _ {\beta / 2} ^ {2}\right) x ^ {2} - \left(2 n \hat {\mu} _ {1} + \frac {Z _ {\beta / 2} ^ {2}}{K}\right) x + n \hat {\mu} _ {1} ^ {2} - \left(\frac {K - 1}{K}\right) Z _ {\beta / 2} ^ {2} \overline {{\mu}} _ {2} = 0.
$$
5: return  $\underline{\mu}_1, \overline{\mu}_1$ .
Proof. Algorithm 4 computes the correct moments  $\pmb{m}_{\ell}^{(i)}$  for  $\ell \leq 4$  due to Proposition 6. Applying Bernstein's inequality to  $\pmb{m}_3$  and  $\pmb{m}_4$ , we get  $\mathbb{P}(\mu_{\ell} \leq \overline{\mu}_{\ell}) \geq 1 - \beta / 4$  for  $\ell = 3, 4$  (see also the proof of Proposition 8).
Next, from Bernstein's inequality applied to  $m_2$ , we have with probability at least  $1 - \beta / 4$  that
$$
\mu_ {2} - \hat {\mu} _ {2} \leq \frac {2}{3 n} \log \frac {4}{\beta} + \sqrt {\frac {2 \sigma_ {2} ^ {2} (\mu_ {2} , \mu_ {3} , \mu_ {4})}{n} \log \frac {4}{\beta}}.
$$
Combining this with the results on  $\overline{\mu}_3, \overline{\mu}_4$  with the union bound, we get with probability at least  $1 - 3\beta / 4$  that
$$
\mu_ {2} - \hat {\boldsymbol {\mu}} _ {2} \leq \frac {2}{3 n} \log \frac {4}{\beta} + \sqrt {\frac {2 \sigma_ {2} ^ {2} (\mu_ {2} , \overline {{\boldsymbol {\mu}}} _ {3} , \overline {{\boldsymbol {\mu}}} _ {4})}{n} \log \frac {4}{\beta}}.
$$
Finally, plugging this into a Bernstein bound on  $m_{1}$  using the variance calculation from (17) (also see the proof of Proposition 9) completes the proof.
C.2 Asymptotic Confidence Intervals
We derive asymptotic versions of the Algorithms 2 to 4 using the Wilson confidence interval.
The Wilson confidence interval is a tightening of the constants for the Bernstein confidence interval
$$
\mu_ {1} - \hat {\mu} _ {1} \leq \sqrt {\frac {2 \log (1 / \beta)}{n} \operatorname {V a r} \left(\boldsymbol {m} _ {1}\right)} + \frac {2}{3 n} \log \frac {1}{\beta} \quad \text {t o} \quad \mu_ {1} - \hat {\mu} _ {1} \leq \sqrt {\frac {Z _ {\beta} ^ {2}}{n} \operatorname {V a r} \left(\boldsymbol {m} _ {1}\right)},
$$
where  $Z_{\beta}$  is the  $(1 - \beta)$ -quantile of the standard Gaussian. Essentially, this completely eliminates the  $1 / n$  term, while the coefficient of the  $1 / \sqrt{n}$  term improves from  $\sqrt{2\log(1 / \beta)}$  to  $Z_{\beta}$  — see Figure 2.
Algorithm 7 Fourth-Order Wilson Intervals
1: For each  $i \in [n]$ , set  $\pmb{m}_1^{(i)} = (1 / K)\sum_{j=1}^K \pmb{x}_j^{(i)}$  and for  $\ell = 1,2,3$ :  $\pmb{m}_{\ell+1}^{(i)} = \pmb{m}_{\ell}^{(i)}\left(\frac{K\pmb{m}_1^{(i)} - \ell}{K - \ell}\right)$ .
2: Set  $\hat{\pmb{\mu}}_{\ell} = (1 / n)\sum_{i = 1}^{n}\pmb{m}_{\ell}^{(i)}$  for  $\ell = 1,2,3,4$
3: For  $\ell = 3,4$ , let  $\overline{\mu}_{\ell}$  be the larger root of the quadratic in  $x$ :
$$
\left(n + Z _ {\beta / 4} ^ {2}\right) x ^ {2} - \left(2 n \hat {\mu} _ {\ell} + Z _ {\beta / 4} ^ {2}\right) x + n \hat {\mu} _ {\ell} ^ {2} = 0.
$$
4: Let  $\overline{\mu}_2$  be the larger root of the quadratic in  $x$ :
$$
\begin{array}{l} \left(n + \frac {2 Z _ {\beta / 4} ^ {2} (2 K - 3)}{K (K - 1)}\right) x ^ {2} - \left(2 n \hat {\mu} _ {2} + \frac {2 Z _ {\beta / 4} ^ {2}}{K (K - 1)}\right) x + n \hat {\mu} _ {2} ^ {2} - c Z _ {\beta / 4} ^ {2} = 0, \\ \text {w h e r e} c = \frac {(K - 2) (K - 3)}{K (K - 1)} \left(\overline {{\boldsymbol {\mu}}} _ {4} - \overline {{\boldsymbol {\mu}}} _ {3} ^ {2}\right) + \frac {4 (K - 2)}{K (K - 1)} \overline {{\boldsymbol {\mu}}} _ {3} \\ \end{array}
$$
5: Set  $\underline{\mu}_1 < \overline{\mu}_1$  as the roots of the quadratic in  $x$ :
$$
\left(n + Z _ {\beta / 4} ^ {2}\right) x ^ {2} - \left(2 n \hat {\mu} _ {1} + \frac {Z _ {\beta / 4} ^ {2}}{K}\right) x + n \hat {\mu} _ {1} ^ {2} - \left(\frac {K - 1}{K}\right) Z _ {\beta / 4} ^ {2} \overline {{\mu}} _ {2} = 0.
$$
The Wilson approximation holds under the assumption that  $(\mu_1 - \hat{\mu}_1) / \sqrt{\mathrm{Var}(\pmb{m}_1) / n}\stackrel {\mathrm{d}}{\approx}\mathcal{N}(0,1)$  and using a Gaussian confidence interval. This can be formalized by the central limit theorem.
Lemma 11 (Lindeberg-Lévy Central Limit Theorem). Consider a sequence of independent random variables  $\pmb{y}^{(1)},\pmb{y}^{(2)},\dots$  with finite moments  $\mathbb{E}[\pmb{y}^{(i)}] = \mu < \infty$  and  $\mathbb{E}(\pmb{y}^{(i)} - \mu)^2 = \sigma^2 < \infty$  for each  $i$ . Then, the empirical mean  $\hat{\pmb{\mu}}_1^{(n)} = (1 / n)\sum_{i = 1}^n\pmb{y}^{(i)}$  based on  $n$  samples satisfies
$$
\lim  _ {n \rightarrow \infty} \mathbb {P} \left(\frac {\hat {\boldsymbol {\mu}} _ {1} ^ {(n)} - \mu}{\sigma / \sqrt {n}} > t\right) = \mathbb {P} _ {\boldsymbol {\xi} \sim \mathcal {N} (0, 1)} (\boldsymbol {\xi} > t)
$$
for all  $t\in \mathbb{R}$  . Consequently, we have,
$$
\lim  _ {n \rightarrow \infty} \mathbb {P} \left(\mu - \hat {\boldsymbol {\mu}} _ {1} ^ {(n)} > \sigma Z _ {\beta} / \sqrt {n}\right) \geq 1 - \beta \quad a n d \quad \lim  _ {n \rightarrow \infty} \mathbb {P} \left(\hat {\boldsymbol {\mu}} _ {1} ^ {(n)} - \mu > \sigma Z _ {\beta} / \sqrt {n}\right) \geq 1 - \beta . \tag {19}
$$
The finite moment requirement above is satisfied in our case because all our random variables are bounded between 0 and 1. The asymptotic confidence interval implied by (19) is known as the Wilson confidence interval. It states that
$$
| \mu - \hat {\boldsymbol {\mu}} _ {1} ^ {(n)} | \leq \frac {\sigma Z _ {\beta / 2}}{\sqrt {n}}.
$$
w.p.  $1 - \beta$  as  $n\to \infty$
We give the Wilson-variants of Algorithms 2 to 4 respectively in Algorithms 5 to 7. Apart from the fact that the Wilson intervals are tighter, we can also solve the equations associated with the Wilson intervals in closed form as they are simply quadratic equations (i.e., without the need for numerical root-finding). The following proposition shows their correctness.
Proposition 12. Consider  $n$  i.i.d. samples  $\pmb{x}^{(1)},\dots ,\pmb{x}^{(n)} \stackrel {i.i.d.}{\sim}\mathsf{XBern}_K(\mu_1,\dots ,\mu_K)$  as inputs to Algorithms 5 to 7. Then, their outputs  $\underline{\mu}_1,\overline{\mu}_1$  satisfy  $\lim_{n\to \infty}\mathbb{P}(\mu_1\leq \underline{\mu}_1)\geq 1 - \beta$  and  $\lim_{n\to \infty}\mathbb{P}(\mu_1\geq \overline{\mu}_1)\geq 1 - \beta$  and  $\lim_{n\to \infty}\mathbb{P}(\underline{\mu}_1\leq \mu_1\leq \overline{\mu}_1)\leq 1 - C\beta$  if
(a)  $K \geq 1$  and  $C = 2$  for Algorithm 5,
(b)  $K \geq 2$  and  $C = 3/2$  for Algorithm 6, and
(c)  $K \geq 4$  and  $C = 5/4$  for Algorithm 7.
We omit the proof as it is identical to those of Propositions 8 to 10 except that it uses the Wilson interval from Lemma 11 rather than the Bernstein interval.
C.3 Scaling of Higher-Order Bernstein Bounds (Proposition 4)
We now re-state and prove Proposition 4.
Proof. We are given  $n$  samples from an XBern distribution  $\pmb{x} \in \{0,1\}^K$  with parameters  $(\mu_1, \dots, \mu_K)$ , where  $\mu_\ell \coloneqq \mathbb{E}[m_\ell]$  with
$$
\boldsymbol {m} _ {\ell} := \frac {1}{K (K - 1) \cdots (K - \ell + 1)} \sum_ {j _ {1} <   j _ {2} <   \dots <   j _ {\ell} \in [ K ]} \boldsymbol {x} _ {j _ {1}} \dots \boldsymbol {x} _ {j _ {\ell}}.
$$
By exchangeability, it also holds that  $\mu_{\ell} = \mathbb{E}[\pmb{x}_1 \cdots \pmb{x}_{\ell}]$ . Assuming a confidence level  $1 - \beta < 1$  and  $K = \lceil n^{(\ell - 1) / \ell} \rceil$ , the 1st-order Bernstein bound gives
$$
\left| \mu_ {1} - \hat {\mu} _ {1} \right| = O \left(\sqrt {\frac {\sigma_ {1} ^ {2}}{n}}\right), \tag {20}
$$
where  $\sigma_{\ell}^{2} \coloneqq \operatorname{Var}(\boldsymbol{m}_{\ell})$ . Expanding  $\sigma_{\ell}^{2}$ , it is easy to show that it is dominated by the 2th-order correlation  $|\mu_{2\ell} - \mu_{\ell}^{2}|$ :
$$
\sigma_ {\ell} ^ {2} = O \left(\frac {1}{K} + | \mu_ {2 \ell} - \mu_ {\ell} ^ {2} |\right) = O \left(\frac {1}{K} + | \hat {\mu} _ {2 \ell} - \hat {\mu} _ {\ell} ^ {2} | + \sqrt {\frac {\sigma_ {2 \ell} ^ {2}}{n}}\right).
$$
Note that our Bernstein confidence interval does not use the fact that the higher-order correlations are small. We only use that assumption to bound the resulting size of the confidence interval in the analysis. Applying the assumption that all the higher order correlations are bounded by  $1 / K$ , i.e.,  $|\mu_{2\ell^{\prime}} - \mu_{\ell^{\prime}}^{2}| = O(1 / K)$ , we get that  $|\hat{\pmb{\mu}}_{2\ell^{\prime}} - \hat{\pmb{\mu}}_{\ell^{\prime}}^{2}| = O(1 / K + \sqrt{\sigma_{2\ell^{\prime}}^{2} / n})$ . Applying this recursively into (20), we get that
$$
\left| \mu_ {1} - \hat {\mu} _ {1} \right| = O \left(\sqrt {\frac {1}{n K}} + \frac {\sigma_ {\ell} ^ {1 / \ell}}{n ^ {(2 \ell - 1) / (2 \ell)}}\right),
$$
for any  $\ell$  that is a power of two. For an  $\ell^{\mathrm{th}}$ -order Bernstein bound, we only use moment estimates up to  $\ell$  and bound  $\sigma_{\ell}^{2} \leq 1$ . The choice of  $K = n^{(\ell - 1) / \ell}$  gives the desired bound:  $|\mu_1 - \hat{\mu}_1| = O(1 / n^{(2\ell - 1) / (2\ell)})$ .
D Canary Design for Lifted DP: Details
The canary design employed in the auditing of the usual  $(\varepsilon, \delta)$ -DP can be easily extended to create distributions over canaries to audit LiDP. We give some examples for common classes of canaries.
Setup. We assume a supervised learning setting with a training dataset  $D_{\mathrm{train}} = \{(x_i, y_i)\}_{i=1}^N$  and a held-out dataset  $D_{\mathrm{val}} = \{(x_i, y_i)\}_{i=N+1}^{N+N'}$  of pairs of input  $x_i \in \mathcal{X}$  and output  $y_i \in \mathcal{Y}$ . We then aim to minimize the average loss
$$
F (\theta) = \frac {1}{N} \sum_ {i = 1} ^ {N} L \left(\left(x _ {i}, y _ {i}\right), \theta\right), \tag {21}
$$
where  $L(z,\theta)$  is the loss incurred by model  $\theta$  on input-output pair  $z = (x,y)$ .
In the presence of canaries  $c_{1},\ldots ,c_{k}$  , we instead aim to minimize the objective
$$
F _ {\text {c a n a r y}} (\theta ; c _ {1}, \dots , c _ {k}) = \frac {1}{N} \left(\sum_ {i = 1} ^ {N} L \left(\left(x _ {i}, y _ {i}\right), \theta\right) + \sum_ {j = 1} ^ {k} L _ {\text {c a n a r y}} \left(c _ {j}, \theta\right)\right), \tag {22}
$$
where  $L_{\mathrm{canary}}(c,\theta)$  is the loss function for a canary  $c$  — this may or may not coincide with the usual loss  $L$ .
Goals of Auditing DP. The usual practice is to set  $D_0 = D_{\mathrm{train}}$  and  $D_1 = D_0 \cup \{c\}$  and  $R \equiv R_c$  for a canary  $c$ . Recall from the definition of  $(\varepsilon, \delta)$ -DP in (1), we have
$$
\varepsilon \geq \sup  _ {c \in \mathcal {C}} \log \left(\frac {\mathbb {P} (\mathcal {A} (D _ {1}) \in R _ {c}) - \delta}{\mathbb {P} (\mathcal {A} (D _ {0}) \in R _ {c})}\right),
$$
for some class  $\mathcal{C}$  of canaries. The goal then is to find canaries  $c$  that approximate the sup over  $\mathcal{C}$ . Since this goal is hard, one usually resorts to finding canaries whose effect can be "easy to detect" in some sense.
Goals of Auditing LiDP. Let  $P_{\text{canary}}$  denote a probability distribution over a set  $\mathcal{C}$  of allowed canaries. We sample  $K$  canaries  $c_1, \ldots, c_K \stackrel{\text{i.i.d.}}{\sim} P_{\text{canary}}$  and set
$$
\boldsymbol {D} _ {0} = D _ {\text {t r a i n}} \cup \left\{\boldsymbol {c} _ {1}, \dots , \boldsymbol {c} _ {K - 1} \right\}, \quad \boldsymbol {D} _ {1} = D _ {\text {t r a i n}} \cup \left\{\boldsymbol {c} _ {1}, \dots , \boldsymbol {c} _ {K} \right\}, \quad \boldsymbol {R} = R _ {\boldsymbol {c} _ {K}}.
$$
From the definition of  $(\varepsilon, \delta)$ -LiDP in (3), we have
$$
\varepsilon \geq \sup  _ {P _ {\text {c a n a r y}}} \log \left(\frac {\mathbb {P} (\mathcal {A} (\boldsymbol {D} _ {1}) \in \boldsymbol {R}) - \delta}{\mathbb {P} (\mathcal {A} (\boldsymbol {D} _ {0}) \in \boldsymbol {R})}\right),
$$
for some class  $\mathcal{C}$  of canaries. The goal then is to approximate the distribution  $P_{\mathrm{canary}}$  for each choice of the canary set  $\mathcal{C}$ . Since this is hard, we will attempt to define a distribution over canaries that are easy to detect (similar to the case of auditing DP). Following the discussion in §3.3, auditing LiDP benefits the most when the canaries are uncorrelated. To this end, we will also impose the restriction that a canary  $c \sim P_{\mathrm{canary}}$ , if included in the training of a model  $\theta$ , is unlikely to change the membership of  $\theta \in R_{c'}$  for an i.i.d. canary  $c' \sim P_{\mathrm{canary}}$  that is independent of  $c$ .
We consider two choices of the canary set  $\mathcal{C}$  (as well as the outcome set  $R_{c}$  and the loss  $L_{\mathrm{canary}}$ ): data poisoning, and random gradients.
D.1 Data Poisoning
We describe the data poisoning approach known as ClipBKD [32] that is based on using the tail singular vectors of the input data matrix and its extension to auditing LiDP.
Let  $X = (x_{1}^{\top}; \dots; x_{N}^{\top}) \in \mathbb{R}^{N \times d}$  denote the matrix with the datapoints  $x_{i} \in \mathbb{R}^{d}$  as rows. Let  $X = \sum_{i=1}^{\min\{N, d\}} \sigma_{i} u_{i} v_{i}^{\top}$  be the singular value decomposition of  $X$  with  $\sigma_{1} \leq \sigma_{2} \leq \dots$  be the singular values arranged in ascending order. Let  $Y$  denote the set of allowed labels.
For this section, we take the set of allowed canaries  $\mathcal{C} = \{\alpha v_1, \alpha v_2, \ldots, \alpha v_{\min\{N,d\}}\} \times \mathcal{Y}$  as the set of the right singular vector of  $X$  scaled by a given factor  $\alpha > 0$  together with any possible target from  $\mathcal{Y}$ . We take  $L_{\mathrm{canary}}(c,\theta) = L(c,\theta)$  to be the usual loss function, and the output set  $R_c$  to be the loss-thresholded set
$$
R _ {c} := \left\{\theta \in \mathcal {R}: L (c, \theta) \leq \tau \right\}, \tag {23}
$$
for some threshold  $\tau$ .
Auditing DP. The ClipBKD approach [32] uses a canary with input  $\alpha v_{1}$ , the singular vector corresponding to the smallest singular value, scaled by a parameter  $\alpha > 0$ . The label is taken as  $y^{\star}(\alpha v_{1})$  where
$$
y^{\star}(x) = \operatorname *{arg  max}_{y\in \mathcal{Y}}L((x,y),\theta_{0}^{\star})
$$
is the target that has the highest loss on input  $x$  under the empirical risk minimizer  $\theta_0^\star = \arg \min_{\theta \in \mathcal{R}} F(\theta)$ . Since a unique  $\theta_0^\star$  is not guaranteed for deep nets nor can we find it exactly,
Figure 7: Comparing the Binomial proportion confidence intervals. We sample  $m \sim \mathrm{Binomial}(n,p)$  for  $p = 0.1$  and  $n$  varying and find the  $95\%$  confidence interval  $[\underline{\mathbf{p}}_n, \overline{\mathbf{p}}_n]$ . We plot the widths  $p - \underline{\mathbf{p}}_n$  and  $\overline{\mathbf{p}}_n - p$  versus  $n$ . We find that all confidence intervals are nearly equivalent once  $n$  is larger than  $\approx 1 / \min \{p, 1 - p\}^2$ .
we train 100 models with different random seeds and pick the class  $y$  that yields the highest average loss over these runs.
Auditing LiDP. We extend ClipBKD to define a probability distribution over a given number  $p$  of canaries. We take
$$
P _ {\text {c a n a r y}} = \operatorname {U n i f o r m} \left(\left\{c _ {1}, \dots , c _ {p} \right\}\right) \quad \text {w i t h} \quad c _ {j} = \left(\alpha v _ {j}, y ^ {\star} (\alpha v _ {j})\right), \tag {24}
$$
i.e.,  $P_{\mathrm{canary}}$  is the uniform distribution over the  $p$  singular vectors corresponding to the smallest singular values.
D.2 Random Gradients
The update poisoning approach of [2] relies on supplying gradients  $c \sim \mathrm{Uniform}(B_{\mathcal{R}}(0,r))$  that are uniform on the Euclidean ball of a given radius  $r$ . This is achieved by setting the loss of the canary as
$$
L _ {\text {c a n a r y}} (c, \theta) = \langle c, \theta \rangle , \quad \text {s o t h a t} \quad \nabla_ {\theta} L _ {\text {c a n a r y}} (c, \theta) = c,
$$
is the desired vector  $c$ .
The set  $R_{c}$  is a threshold of the dot product
$$
R _ {c} = \{\theta \in \mathcal {R}: \langle c, \theta \rangle \leq \tau \} \tag {25}
$$
for a given threshold  $\tau$ . This set is analogous to the loss-based thresholding of (23) in that both can be written as  $L_{\mathrm{canary}}(c,\theta)\leq \tau$ .
Auditing DP and LiDP. The random gradient approach of [2] relies on defining a distribution  $P_{\mathrm{canary}} \equiv \mathrm{Uniform}(B_{\mathcal{R}}(0,r))$  over canaries. It can be used directly to audit LiDP.
E Simulations with the Gaussian Mechanism: Details and More Results
Here, we give the full details and additional results of auditing the Gaussian mechanism with synthetic data in  $\S 4$ .
Fix a dimension  $d$  and a failure probability  $\beta \in (0,1)$ . Suppose we have a randomized algorithm  $\mathcal{A}$  that returns a noisy sum of its inputs with the goal of simulating the Gaussian mechanism. Concretely, the input space  $\mathcal{Z} = \{z\in \mathbb{R}^d:\| z\| _2\leq 1\}$  is the unit ball in  $\mathbb{R}^d$ . Given a finite set  $D\in \mathcal{Z}^*$ , we sample a vector  $\xi \sim \mathcal{N}(0,\sigma^2 I_d)$  of a given variance  $\sigma^2$  and return
$$
\mathcal {A} (D) = \boldsymbol {\xi} + \sum_ {z \in D} z.
$$
Figure 8: Effect of the number  $n$  of trials on the empirical lower bound  $\hat{\varepsilon}$  from auditing the Gaussian mechanism for DP and LiDP. The shaded are denotes the standard error over 25 random seeds.
To isolate the effect of the canaries, we set our original dataset  $D = \{\mathbf{0}_d\}$  as a singleton with the vector of zeros in  $\mathbb{R}^d$ . Since we are in the blackbox setting, we do not assume that this is known to the auditor.
DP Upper Bound. The non-private version of our function computes the sum  $D \mapsto \sum_{z \in D} z$ . where each  $z \in D$  is a canary. Hence, the  $\ell_2$  sensitivity of the operation is  $\Delta_2 = \max_{x \in D} \| x \|_2 = 1$ , as stated in §4.
Since we add  $\pmb{\xi} \sim \mathcal{N}(0, \sigma^2 I_d)$ , it follows that the operation  $\mathcal{A}(\cdot)$  is  $(\alpha, \alpha / (2\sigma^2))$ -RDP for every  $\alpha > 1$ . Thus,  $\mathcal{A}(\cdot)$  is  $(\varepsilon_{\delta}, \delta)$ -DP where
$$
\varepsilon_ {\delta} \leq \inf  _ {\alpha > 1} \left\{\frac {\alpha}{2 \sigma^ {2}} + \frac {1}{\alpha - 1} \log \frac {1}{\alpha \delta} + \log \left(1 - \frac {1}{\alpha}\right) \right\},
$$