SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Lemma A.1. Let  $\mathbf{B}^0$ ,  $\mathbf{B}^1$  be two distributions. Then  $\mathbf{B}^0 \approx_{\varepsilon, \delta} \mathbf{B}^1$  if and only if we can express them as mixtures
$$
\mathbf {B} ^ {b} \equiv (1 - \delta) \cdot \mathbf {N} ^ {b} + \delta \cdot \mathbf {E} ^ {b},
$$
where  $\mathbf{N}^0\approx_{\varepsilon}\mathbf{N}^1$
We treat random variables interchangeably as distributions, and in particular, for a randomized algorithms  $\mathcal{A}$  and input  $D$  we use  $\mathcal{A}(D)$  to denote both the random variable and the distribution.
For a space of datasets equipped with a symmetric neighboring relation, we say an algorithm  $\mathcal{A}$  is  $\varepsilon$ -DP (pure differential privacy), if for any two neighboring datasets  $D$  and  $D'$ ,  $\mathcal{A}(D) \approx_{\varepsilon} \mathcal{A}(D')$ . Similarly, we say  $\mathcal{A}$  is  $(\varepsilon, \delta)$ -DP (approximate differential privacy) if for any two neighboring datasets  $D, D'$ , it holds that  $\mathcal{A}(D) \approx_{\varepsilon, \delta} \mathcal{A}(D')$  [11]. We refer to  $\varepsilon, \delta$  as the privacy parameters.
A private test is a differentially private algorithm with Boolean output (say in  $\{0,1\}$ ).
Remark A.2. The literature in differential privacy uses different definitions of neighboring datasets. TCT, and properties in these preliminaries, apply with any symmetric relation over the space of possible input datasets.
The following is immediate from Lemma A.1:
Corollary A.3 (Decomposition of an approximate DP Algorithm). An algorithm  $\mathcal{A}$  is  $(\varepsilon, \delta)$ -DP if and only if for any two neighboring datasets  $D^0$  and  $D^1$  we can represent each distribution  $\mathcal{A}(D^b)$ $(b \in \{0, 1\})$  as a mixture
$$
\mathcal {A} (D ^ {b}) \equiv (1 - \delta) \cdot \mathbf {N} ^ {b} + \delta \cdot \mathbf {E} ^ {b},
$$
Differential privacy satisfies the post-processing property (post-processing of the output of a private algorithm remains private with the same parameter values) and also has nice composition theorems:
Lemma A.4 (DP composition [11, 14]). An interactive sequence of  $r$  executions of  $\varepsilon$ -DP algorithms satisfies  $(\varepsilon', \delta)$ -DP for
-  $\varepsilon' = r\varepsilon$  and  $\delta = 0$  by basic composition [11], or
for any  $\delta >0$
$$
\varepsilon^ {\prime} = \frac {1}{2} r \varepsilon^ {2} + \varepsilon \sqrt {2 r \log (1 / \delta)}.
$$
by advanced composition [14].
A.1 Simulation-based privacy analysis
Privacy analysis of an algorithm  $\mathcal{A}$  via simulations is performed by simulating the original algorithm  $\mathcal{A}$  on two neighboring datasets  $D^0, D^1$ . The simulator does not know which of the datasets is the actual input (but knows everything about the datasets). Another entity called the "data holder" has the 1-bit information  $b \in \{0, 1\}$  on which dataset it is. We perform privacy analysis with respect to what the holder discloses to the simulator regarding the private bit  $b$  (taking the maximum over all choices of  $D^0, D^1$ ). The privacy analysis is worst case over the choices of two neighboring datasets. This is equivalent to performing privacy analysis for  $\mathcal{A}$ .
Lemma A.5 (Simulation-based privacy analysis). [8] Let  $\mathcal{A}$  be an algorithm whose input is a dataset. If there exist a pair of interactive algorithms  $S$  and  $H$  satisfying the following 2 properties, then algorithm  $\mathcal{A}$  is  $(\varepsilon, \delta)$ -DP.
1. For every two neighboring datasets  $D^0, D^1$  and for every bit  $b \in \{0, 1\}$  it holds that
$$
\left(\mathcal {S} \left(D ^ {0}, D ^ {1}\right) \leftrightarrow H \left(D ^ {0}, D ^ {1}, b\right)\right) \equiv \mathcal {A} \left(D ^ {b}\right).
$$
Here  $\left(S(D^0,D^1)\leftrightarrow H(D^0,D^1,b)\right)$  denotes the outcome of  $S$  after interacting with  $H$ .
2. Algorithm  $H$  is  $(\varepsilon, \delta)$ -DP w.r.t. the input bit  $b$ .
A.2 Privacy Analysis with Failure Events
Privacy analysis of a randomized algorithm  $\mathcal{A}$  using designated failure events is as follows:
1. Designate some runs of the algorithm as failure events.
2. Compute an upper bound on the maximum probability, over datasets  $D$ , of a transcript with a failure designation.
3. Analyse the privacy of the interaction transcript conditioned on no failure designation.
Note that the failure designation is only used for the purpose of analysis. The output on failure runs is not restricted (e.g., could be the dataset  $D$ )
Lemma A.6 (Privacy analysis with privacy failure events). Consider privacy analysis of  $\mathcal{A}$  with failure events. If the probability of a failure event is bounded by  $\delta^{*} \in [0,1]$  and the transcript conditioned on non-failure is  $(\varepsilon', \delta')$ -DP then the algorithm  $\mathcal{A}$  is  $(\varepsilon, \delta + \delta^{*})$ -DP.
Proof. Let  $D^0$  and  $D^1$  be neighboring datasets. From our assumptions, for  $b \in \{0, 1\}$ , we can represent  $\mathcal{A}(D^b)$  as the mixture  $\mathcal{A}(D^b) \equiv (1 - \delta^b) \cdot \mathbf{Z}^b + \delta^b \cdot \mathbf{F}^b$ , where  $\mathbf{Z}^0 \approx_{\varepsilon', \delta'} \mathbf{Z}^1$ , and  $\delta^{(b)} \leq \delta^*$ . From Lemma A.1, we have  $\mathbf{Z}^b \equiv (1 - \delta') \cdot \mathbf{N}^b + \delta' \cdot \mathbf{E}^b$ , where  $\mathbf{N}^0 \approx_{\varepsilon'} \mathbf{N}^1$ .
$$
\begin{array}{l} \mathcal {A} (D ^ {b}) = \left(1 - \delta^ {(b)}\right) \cdot \mathbf {Z} ^ {b} + \delta^ {(b)} \cdot \mathbf {F} ^ {(b)} \\ = (1 - \delta^ {*}) \cdot \mathbf {Z} ^ {b} + (\delta^ {*} - \delta^ {(b)}) \cdot \mathbf {Z} ^ {b} + \delta^ {(b)} \cdot \mathbf {F} ^ {b} \\ = (1 - \delta^ {*}) \cdot \mathbf {Z} ^ {b} + \delta^ {*} \cdot \left((1 - \delta^ {(b)} / \delta^ {*}) \cdot \mathbf {Z} ^ {b} + \delta^ {(b)} \cdot \mathbf {F} ^ {b}\right) \\ = (1 - \delta^ {*}) (1 - \delta^ {\prime}) \cdot \mathbf {N} ^ {b} + (1 - \delta^ {*}) \delta^ {\prime} \cdot \mathbf {E} ^ {b} + \delta^ {*} \cdot \left((1 - \delta^ {(b)} / \delta^ {*}) \cdot \mathbf {Z} ^ {b} + \delta^ {(b)} \cdot \mathbf {F} ^ {b}\right) \\ = \left(1 - \delta^ {*} - \delta^ {\prime}\right) \cdot \mathbf {N} ^ {b} + \delta^ {\prime} \delta^ {*} \cdot \mathbf {N} + \left(1 - \delta^ {*}\right) \delta^ {\prime} \cdot \mathbf {E} ^ {b} + \delta^ {*} \cdot \left(\left(1 - \delta^ {(b)} / \delta^ {*}\right) \cdot \mathbf {Z} ^ {b} + \delta^ {(b)} \cdot \mathbf {F} ^ {b}\right) \\ \end{array}
$$
The claim follows from Corollary A.3.
Using simulation-based privacy analysis we can treat an interactive sequence of approximate-DP algorithms (optionally with designated failure events) as a respective interactive sequence of pure-DP algorithms where the  $\delta$  parameters are anlayed through failure events. This simplifies analysis:
We can relate the privacy of a composition of approximate-DP algorithms to that of a composition of corresponding pure-DP algorithms:
Corollary A.7 (Composition of approximate-DP algorithms). An interactive sequence of  $(\varepsilon_i, \delta_i)$ -DP algorithms  $(i \in [k])$  has privacy parameter values  $(\varepsilon', \delta' + \sum_{i=1}^{k} \delta_i)$ , where  $(\varepsilon', \delta')$  are privacy parameter values of a composition of pure  $(\varepsilon_i, 0)$ -DP algorithms  $i \in [k]$ .
Proof. We perform simulation-based analysis. Fix two neighboring datasets  $D^0, D^1$ . For an  $(\varepsilon_i, \delta_i)$ -DP algorithm, we can consider the mixtures as in Corollary A.3. We draw  $c \sim \mathbf{Ber}(\delta_i)$  and if  $c = 1$  designate the output as failure and return  $r \sim \mathbf{E}^{(b)}$ . Otherwise, we return  $r \sim \mathbf{N}^{(b)}$ . The overall failure probability is bounded by  $1 - \prod_{i}(1 - \delta_{i}) \leq \sum_{i} \delta_{i}$ . The output conditioned on non-failure is a composition of  $(\varepsilon_i, 0)$ -DP algorithms  $(i \in [k])$ . The claim follows using Lemma A.6.
B The Target-Charging Technique
We extend the definition of  $q$ -targets (Definition 2.1) so that it applies with approximate DP algorithms:
Definition B.1 ( $q$ -target with  $(\varepsilon, \delta)$  of a pair of distributions). Let  $\mathcal{A} \to \mathcal{Y}$  be a randomized algorithm. Let  $\mathbf{Z}^0$  and  $\mathbf{Z}^1$  be two distributions with support  $\mathcal{Y}$ . We say that  $\top \subseteq \mathcal{Y}$  is a  $q$ -target of  $(\mathbf{Z}^0, \mathbf{Z}^1)$  with  $(\varepsilon, \delta)$ , where  $\varepsilon > 0$  and  $\delta \in [0,1)$ , if there exist  $p \in [0,1]$  and five distributions  $\mathbf{C}$ ,  $\mathbf{B}^b$ , and  $\mathbf{E}^b$  (for  $b \in \{0,1\}$ ) such that  $\mathbf{Z}^0$  and  $\mathbf{Z}^1$  can be written as the mixtures
$$
\mathbf {Z} ^ {0} \equiv (1 - \delta) \cdot (p \cdot \mathbf {C} + (1 - p) \cdot \mathbf {B} ^ {0}) + \delta \cdot \mathbf {E} ^ {0}
$$
$$
\mathbf {Z} ^ {1} \equiv (1 - \delta) \cdot (p \cdot \mathbf {C} + (1 - p) \cdot \mathbf {B} ^ {1}) + \delta \cdot \mathbf {E} ^ {1}
$$
where  $\mathbf{B}^0\approx_{\varepsilon}\mathbf{B}^1$  , and min  $(\operatorname *{Pr}[\mathbf{B}^0\in \mathsf{T}],\operatorname *{Pr}[\mathbf{B}^1\in \mathsf{T}])\geq q.$
Definition B.2 ( $q$ -target with  $(\varepsilon, \delta)$  of a randomized algorithm). Let  $\mathcal{A} \to \mathcal{Y}$  be a randomized algorithm. We say that  $\top \subseteq \mathcal{Y}$  is a  $q$ -target of  $\mathcal{A}$  with  $(\varepsilon, \delta)$ , where  $\varepsilon > 0$  and  $\delta \in [0,1)$ , if for any pair  $D^0, D^1$  of neighboring datasets,  $\top$  is a  $q$ -target with  $(\varepsilon, \delta)$  of  $\mathcal{A}(D^0)$  and  $\mathcal{A}(D^1)$ .
We can relate privacy of an algorithms or indistinguishability of two distributions to existence of  $q$ -targets:
Lemma B.3. (i) If  $(\mathbf{Z}^0, \mathbf{Z}^1)$  have a  $q$ -target with  $(\varepsilon, \delta)$  then  $\mathbf{Z}^0 \approx_{\varepsilon, \delta} \mathbf{Z}^1$ . Conversely, if  $\mathbf{Z}^0 \approx_{\varepsilon, \delta} \mathbf{Z}^1$  then  $(\mathbf{Z}^0, \mathbf{Z}^1)$  have a 1-target with  $(\varepsilon, \delta)$  (the full support is a 1-target).
(ii) If an algorithm  $\mathcal{A}$  has a  $q$ -target with  $(\varepsilon, \delta)$  then  $\mathcal{A}$  is  $(\varepsilon, \delta)$ -DP. Conversely, if an algorithm  $\mathcal{A}$  is  $(\varepsilon, \delta)$ -DP then it has a 1-target (the set  $\mathcal{Y}$ ) with  $(\varepsilon, \delta)$ .
Proof. If two distributions  $\mathbf{B}^0, \mathbf{B}^1$  have a  $q$ -target with  $(\varepsilon, \delta)$  than from Definition B.1 they can be represented as mixtures. Now observe the if  $\mathbf{B}^0 \approx_{\varepsilon} \mathbf{B}^1$  then the mixtures also satisfy  $p \cdot \mathbf{C} + (1 - p) \cdot \mathbf{B}^0 \approx_{\varepsilon} p \cdot \mathbf{C} + (1 - p) \cdot \mathbf{B}^0$ . Using Lemma A.1, we get  $\mathbf{Z}^0 \approx_{\varepsilon, \delta} \mathbf{Z}^1$ .
For (ii) consider  $\mathcal{A}$  and two neighboring datasets  $D^0$  and  $D^{1}$ . Using Definition B.2 and applying the argument above we obtain  $\mathcal{A}(\bar{D}^0)\approx_{\varepsilon ,\delta}\mathcal{A}(D^1)$ . The claim follows using Corollary A.3.
Now for the converse. If  $\mathbf{Z}^0\approx_{\varepsilon ,\delta}\mathbf{Z}^1$  then consider the decomposition as in Lemma A.1. Now we set  $p = 0$  and  $\mathbf{B}^b\gets \mathbf{N}^b$  to obtain the claim with  $q = 1$  and the target being the full support.
For (ii), if  $\mathcal{A} \to \mathcal{Y}$  is  $(\varepsilon, \delta)$ -DP then consider neighboring  $\{D^0, D^1\}$ . We have  $\mathcal{A}(D^0) \approx_{\varepsilon, \delta} \mathcal{A}(D^1)$ . We proceed as with the distributions.
Algorithm 6 is an extension of Algorithm 1 that permits calls to approximate DP algorithms. The extension also inputs a bound  $\tau$  on the number of target hits and a bound  $\tau_{\delta}$  on the cumulative  $\delta$  parameter values of the algorithms that were called. We apply adaptively a sequence of  $(\varepsilon, \delta)$ -DP algorithms with specified  $q$ -targets to the input data set  $D$  and publish the results. We halt when the first of the following happens (1) the respective target sets are hit for a specified  $\tau$  number of times (2) the accumulated  $\delta$ -values exceed the specified limit  $\tau_{\delta}$ .
The privacy cost of Target-Charging is as follows (This is a precise and more general statement of Lemma 2.2):
Algorithm 6: Target Charging with Approximate DP
Input: Dataset  $D = \{x_{1},\ldots ,x_{n}\} \in X^{n}$  .Integer  $\tau \geq 1$  (Upper limit on the number of target hits).  $\tau_{\delta}\geq 0$  (upper limit on cumulative  $\delta$  parameter). Fraction  $q\in [0,1]$ $C\gets 0,C_{\delta}\gets 0$  // Initialize target hit and failure counters   
for  $i = 1,\dots$  do // Main loop   
Receive  $(A_{i},\top_{i})$  where  $A_{i}$  is an  $(\varepsilon ,\delta_i)$  -DP mechanism, and  $\top_{i}$  is a q-target with  $(\varepsilon ,\delta_{i})$  for  $\mathcal{A}$ $r\gets A_i(D)$    
if  $C_\delta +\delta_i > \tau_\delta$  then Halt   
 $C_{\delta}\leftarrow C_{\delta} + \delta$  // TCT charge for  $\delta_{i}$    
Publish r   
if  $r\in \top$  then // TCT Charge for a q-target hit with  $\varepsilon$ $\begin{array}{rl} & C\gets C + 1\\ & \text{if} C = \tau \text{then Halt} \end{array}$
Algorithm 7: Simulation of Target Charging
Input: Two neighboring datasets  $D^0, D^1$ , private  $b \in \{0, 1\}$ ,  $\tau \in \mathbb{N}$ ,  $\tau_{\delta} \in \mathbb{R}_{\geq 0}$ ,  $q \in [0, 1]$ ,  $\alpha > 0$ .  
 $C \leftarrow 0, C_{\delta} \leftarrow 0, h \leftarrow 0$  // Initialize;  $h$  is a counter on the number of non-fail calls to data holder  
for  $i = 1, \ldots, \dots$  do // Main loop  
Receive  $(A_i, \top_i)$  where  $A_i$  is an  $(\varepsilon, \delta_i)$ -DP mechanism, and  $\top_i$  is a  $q$ -target with  $(\varepsilon, \delta_i)$  for  $A$   
if  $C_{\delta} + \delta_i > \tau_{\delta}$  then Halt  
 $C_{\delta} \leftarrow C_{\delta} + \delta$   
Let  $p \in [0, 1]$ ,  $\mathbf{C}, \mathbf{B}^0 \approx_{\varepsilon} \mathbf{B}^1$ , and  $\mathbf{E}^b$  (for  $b \in \{0, 1\}$ ) such that  
\[
\begin{array}{l}
A(D^b) \equiv (1 - \delta) \cdot (p \cdot \mathbf{C} + (1 - p) \cdot \mathbf{B}^b) + \delta \cdot \mathbf{E}^b
\end{array}
// By Definition B.2  
if Ber( $\delta$ )  $\equiv 1$  then  
Fail  
Publish  $r \sim \mathbf{E}^b$   
else  
if Ber( $p$ )  $\equiv 1$  then  
Publish  $r \sim \mathbf{C}$  // No access to data holder  
else  
Publish  $r \sim \mathbf{B}^b$  // ε-DP Data Holder Call  
 $h \leftarrow h + 1$  // counter of ε-private data holder calls  
if  $h > (1 + \alpha)\tau / q$  then // Number of Holder calls exceeded limit  
 $\sqsubset$  Fail  
if  $r \in T$  then  
 $C \leftarrow C + 1$  // outcome is a target hit  
if  $C = \tau$  then Halt
Theorem B.4 (Privacy of Target-Charging). Algorithm 6 satisfies the following approximate DP privacy bounds:
$$
\left(\left(1 + \alpha\right) \frac {\tau}{q} \varepsilon , C _ {\delta} + \delta^ {*} (\tau , \alpha)\right), \quad \text {f o r a n y} \alpha > 0;
$$
$$
\left(\frac {1}{2} (1 + \alpha) \frac {\tau}{q} \varepsilon^ {2} + \varepsilon \sqrt {(1 + \alpha) \frac {\tau}{q} \log (1 / \delta)}, \delta + C _ {\delta} + \delta^ {*} (\tau , \alpha)\right), \quad f o r a n y \delta > 0, \alpha > 0.
$$
where  $\delta^{*}(\tau, \alpha) \leq e^{-\frac{\alpha^{2}}{2(1 + \alpha)}\tau}$  and  $C_{\delta} \leq \tau_{\delta}$  is as computed by the algorithm.
Proof. We apply the simulation-based privacy analysis in Lemma A.5 and use privacy analysis with failure events (Lemma A.6).
The simulation is described in Algorithm 7. Fix two neighboring data sets  $D^0$  and  $D^1$ . The simulator initializes the target hit counter  $C \gets 0$  and the cumulative  $\delta$ -values tracker  $C_{\delta} \gets 0$ . For  $i \geq 1$  it proceeds as follows. It receives  $(\mathcal{A}_i, \top_i)$  where  $\mathcal{A}_i$  is  $(\varepsilon, \delta_i)$ -DP. If  $C_{\delta} + \delta_i > \tau_{\delta}$  it halts. Since  $\top_i$  is a  $q$ -target for  $\mathcal{A}_i$ , there are  $p, \mathbf{C}, \mathbf{B}^0, \mathbf{B}^1, \mathbf{E}^0$  and  $\mathbf{E}^1$  as in Definition B.2. The simulator flips a biased coin  $c' \sim \mathbf{Ber}(\delta)$ . If  $c' = 1$  it outputs  $r \sim \mathbf{E}^b$  and the execution is designated as **Fail**. In this
case there is an interaction with the data holder but also a failure designation. The simulator flips a biased coin  $c \sim \mathbf{Ber}(p)$ . If  $c = 1$ , then the simulator publishes a sample  $r \sim \mathbf{C}$  (this does not require an interaction with the data holder). Otherwise, the data holder is called. The data holder publishes  $r \sim \mathbf{B}^{\mathbf{b}}$ . We track the number  $h$  of calls to the data holder. If  $h$  exceeds  $(1 + \alpha)\tau / q$ , we designate the execution as Fail. If  $r \in \top_{i}$  then  $C$  is incremented. If  $C = \tau$ , the algorithm halts.
The correctness of the simulation (faithfully simulating Algorithm 1 on the dataset  $D^{b}$ ) is straightforward. We analyse the privacy cost. We will show that
(i) the simulation designated a failure with probability at most  $C_{\delta} + \delta^{*}(\tau, \alpha)$ .
(ii) Conditioned on no failure designation, the simulation performed at most  $r = (1 + \alpha)\frac{\tau}{q}$  adaptive calls to  $(\varepsilon, 0)$ -DP algorithms
Observe that (ii) is immediate from the simulation declaring failure when  $h > r$ . We will establish (i) below.
The statement of the Theorem follows from Lemma A.6 and when applying the DP composition bounds (Lemma A.4). The first bounds follow using basic composition and the second follow using advanced composition [14].
This analysis yields the claimed privacy bounds with respect to the private bit  $b$ . From Lemma A.5 this is the privacy cost of the algorithm.
It remains to show bound the failure probability. There are two ways in which a failure can occur. The first is on each call, with probability  $\delta_{i}$ . This probability is bounded by  $1 - \prod_{i} \delta_{i} \leq \sum_{i} \delta_{i} \leq C_{\delta}$ . The second is when the number  $h$  of private accesses to the data holder exceeds the limit. We show that the probability that the algorithm halts with failure due to that is at most  $\delta^{*}$ .
We consider a process that continues until  $\tau$  charges are made. The privacy cost of the simulation (with respect to the private bit  $b$ ) depends on the number of times that the data holder is called. Let  $X$  be the random variable that is the number of calls to the data holder. Each call is  $\varepsilon$ -DP with respect to the private  $b$ . In each call, there is probability at least  $q$  for a "charge" (increment of  $C$ ).
A failure is the event that the number of calls to data holder exceeds  $(1 + \alpha)\tau / q$  before  $\tau$  charges are made. We show that this occurs with probability at most  $\delta^{*}(\tau, \alpha)$ :
$$
\Pr \left[ X > (1 + \alpha) \frac {\tau}{q} \right] \leq \delta^ {*} (\tau , \alpha). \tag {1}
$$
To establish (1), we first observe that the distribution of the random variable  $X$  is dominated by a random variable  $X'$  that corresponds to a process of drawing i.i.d.  $\mathbf{Ber}(q)$  until we get  $\tau$  successes (Domination means that for all  $m$ ,  $\operatorname*{Pr}[X' > m] \geq \operatorname*{Pr}[X > m]$ ). Therefore, it suffices to establish that
$$
\Pr \left[ X ^ {\prime} > (1 + \alpha) \frac {\tau}{q} \right] \leq \delta^ {*} (\tau , \alpha).
$$
Let  $Y$  be the random variable that is a sum of  $m = 1 + \left\lfloor (1 + \alpha)\frac{\tau}{q}\right\rfloor$  i.i.d.  $\mathbf{Ber}(q)$  random variables. Note that
$$
\operatorname * {P r} \left[ X ^ {\prime} > (1 + \alpha) \frac {\tau}{q} \right] = \operatorname * {P r} [ Y <   \tau ].
$$
We bound  $\operatorname*{Pr}[Y < \tau]$  using multiplicative Chernoff bounds [4]. The expectation is  $\mu = mq$  and we bound the probability that the sum of Bernoulli random variables is below  $\frac{1}{1 + \alpha}\mu = (1 - \frac{\alpha}{1 + \alpha})\mu$ . Using the simpler form of the bounds we get using  $\mu = mq \geq (1 + \alpha)\tau$
$$
\Pr [ Y <   \tau ] = \Pr [ Y <   (1 - \frac {\alpha}{1 + \alpha}) \mu ] \leq e ^ {- \frac {\alpha^ {2}}{2 (1 + \alpha) ^ {2}} \mu} \leq e ^ {- \frac {\alpha^ {2}}{2 (1 + \alpha)} \tau}.
$$
<sup>1</sup>Bound can be tightened when using precise tail probability values.
Remark B.5 (Number of target hits). The TCT privacy analysis has a tradeoff between the final “ $\varepsilon$ ” and “ $\delta$ ” privacy parameters. There is multiplicative factor of  $(1 + \alpha)$  ( $\sqrt{1 + \alpha}$  with advanced composition) on the “ $\varepsilon$ ” privacy parameter. But when we use a smaller  $\alpha$  we need a larger value of  $\tau$  to keep the “ $\delta$ ” privacy parameter small. For a given  $\alpha, \delta^{*} > 0$ , we can calculate a bound on the smallest value of  $\tau$  that works. We get
$$
\tau \geq 2 \frac {1 + \alpha}{\alpha^ {2}} \cdot \ln (1 / \delta^ {*}) \quad \text {(s i m p l i f i e d C h e r n o f f)}
$$
$$
\tau \geq \frac {1}{(1 + \alpha) \ln \left(e ^ {\alpha / (1 + \alpha)} (1 + \alpha) ^ {- 1 / (1 + \alpha)}\right)} \cdot \ln \left(1 / \delta^ {*}\right) \quad \text {(r a w C h e r n o f f)}
$$
For  $\alpha = 0.5$  we get  $\tau > 10.6 \cdot \ln(1 / \delta^{*})$ . For  $\alpha = 1$  we get  $\tau > 3.26 \cdot \ln(1 / \delta^{*})$ . For  $\alpha = 5$  we get  $\tau > 0.31 \cdot \ln(1 / \delta^{*})$ .
Remark B.6 (Mix-and-match TCT). TCT analysis can be extended to the case where we use algorithms with varied privacy guarantees  $\varepsilon_{i}$  and varied  $q_{i}$  values. In this case the privacy cost depends on  $\sum_{i|A_i(D)\in \mathsf{T}_i}\frac{\varepsilon_i}{q_i}$ . The analysis relies on tail bounds on the sum of random variables, is more complex. Varied  $\varepsilon$  values means the random variables have different size supports. A simple coarse bound is according to the largest support, which allows us to use a simple counter for target hits, but may be lossy with respect to precise bounds. The discussion concerns the (analytical or numerical) derivation of tail bounds in non-specific to TCT and is tangential to our contribution.
B.1 Multi-Target TCT
Multi-target charging is described in Algorithm 8. We show the following
Lemma B.7 (Privacy of multi-TCT). Algorithm 8 satisfies  $(\varepsilon', k\delta')$ -approximate DP bounds, where  $(\varepsilon', \delta')$  are privacy bounds for single-target charging (Algorithm 1).
Specifically, when we expect that one (index) of multiple outcomes  $\perp_1,\dots ,\perp_k$  will dominate our interaction but can not specify which one it is in advance, we can use  $k$ -TCT with NotPrior targets with priors  $\perp_{1},\ldots ,\perp_{k}$ . From Lemma B.7, the overall privacy cost depends on the number of times that the reported output is different than the most dominant outcome. More specifically, for private testing, when we expect that one type of outcome would dominate the sequence but we do not know if it is 0 or 1, we can apply 2-TCT. The total number of target hits corresponds to the less dominant outcome. The total number of privacy charges (on average) is at most (approximately for small  $\varepsilon$ ) double that, and therefore is always comparable or better to composition (can be vastly lower when there is a dominant outcome).
Algorithm 8: Multi-Target Charging
Input: Dataset  $D = \{x_{1},\ldots ,x_{n}\} \in X^{n}$  .Integer  $\tau \geq 1$  (charging limit). Fraction  $q\in [0,1]$ $k\geq 1$  (number of targets).   
for  $i\in [k]$  do  $C_i\gets 0$  // Initialize charge counters   
while  $\min_{i\in [k]}C_i <   \tau$  do // Main loop   
Receive  $(A,(T_{i})_{i\in [k]})$  where  $\mathcal{A}$  is an  $\varepsilon$  -DP mechanism, and  $\top_{i}$  is a q-target for A   
 $r\gets \mathcal{A}(D)$    
Publish r   
for  $i\in [k]$  do   
if  $r\in T_i$  then  $C_i\gets C_i + 1$  // outcome is in q-target  $\top_{i}$
Proof of Lemma B.7 (Privacy of multi-Target TCT).  ${}^{3}$  Let  $\left( {\varepsilon ,\delta }\right)$  be the privacy bounds for  ${\mathcal{M}}_{i}$  that is single-target TCT with  $\left( {{\mathcal{A}}_{i},{\top }_{i}}\right)$  . Let  $\mathcal{M}$  be the  $k$  -target algorithm. Let  ${\top }_{i}^{j}$  be the  $i$  th target in step  $j$  .