SlowGuess/NeurIPS_2023_OCR huggingface dataset:
One of our applications of revise calls to conditional release (see Section D applies TCT with both  $\varepsilon$ -DP and  $2\varepsilon$ -DP algorithms even for base  $\varepsilon$ -DP algorithm)
<sup>3</sup>We note that the claim generally holds for online privacy analysis with the best of multiple methods. We provide a proof specific to multi-target charging below.
We say that an outcome sequence  $R = (r_j)_{j=1}^h \in R$  is valid for  $i \in [k]$  if and only if  $\mathcal{M}_i$  would halt with this output sequence, that is,  $\sum_{j=1}^h \mathbf{1}\{r_j \in \mathsf{T}_i^j\} = \tau$  and  $r_h \in \mathsf{T}_i^h$ . We define  $G(R) \subset [k]$  to be all  $i \in [k]$  for which  $R$  is valid.
Consider a set of sequences  $H$ . Partition  $H$  into  $k + 1$  sets  $H_{i}$  so that  $H_0 = \{R\in H\mid G(R) = \emptyset \}$  and  $H_{i}$  may only include  $R\in H$  for which  $i\in G(R)$ . That is,  $H_0$  contains all sequences that are not valid for any  $i$  and  $H_{i}$  may contain only sequences that are valid for  $i$ .
$$
\begin{array}{l} \operatorname * {P r} [ \mathcal {M} (D) \in H ] = \sum_ {i = 1} ^ {k} \operatorname * {P r} [ \mathcal {M} (D) \in H _ {i} ] = \sum_ {i = 1} ^ {k} \operatorname * {P r} [ \mathcal {M} _ {i} (D) \in H _ {i} ] \\ \leq \sum_ {i = 1} ^ {k} \left(e ^ {\varepsilon} \cdot \Pr [ \mathcal {M} _ {i} (D ^ {\prime}) \in H _ {i} ] + \delta\right) = e ^ {\varepsilon} \cdot \sum_ {i = 1} ^ {k} \Pr [ \mathcal {M} _ {i} (D ^ {\prime}) \in H _ {i} ] + k \cdot \delta \\ = e ^ {\varepsilon} \Pr [ \mathcal {M} (D ^ {\prime}) \in H ] + k \cdot \delta . \\ \end{array}
$$
C Properties of NotPrior targets
Recall that a NotPrior target of an  $(\varepsilon, \delta)$ -DP algorithm is specified by any potential outcome (of our choice) that we denote by  $\perp$ . The NotPrior target is the set of all outcomes except  $\perp$ . In this Section we prove (a more general statement of) Lemma 2.3:
Lemma C.1 (Property of a NotPrior target). Let  $\mathcal{M}: X \to \mathcal{Y} \cup \{\bot\}$ , where  $\bot \notin \mathcal{V}$ , be an  $(\varepsilon, \delta)$ -DP algorithm. Then the set of outcomes  $\mathcal{V}$  constitutes an  $\frac{1}{e^{\varepsilon} + 1}$ -target with  $(\varepsilon, \delta)$  for  $\mathcal{M}$ .
We will use the following lemma:
Lemma C.2. If two distributions  $\mathbf{Z}^0$ ,  $\mathbf{Z}^1$  with support  $\mathcal{Y} \cup \{\bot\}$  satisfy  $\mathbf{Z}^0 \approx_{\varepsilon} \mathbf{Z}^1$  then  $\mathcal{Y}$  constitutes an  $\frac{1}{e^{\varepsilon} + 1}$ -target with  $(\varepsilon, 0)$  for  $(\mathbf{Z}^0, \mathbf{Z}^1)$ .
Proof of Lemma 2.3. From Definition B.2, it suffices to show that for any two neighboring datasets,  $D^0$  and  $D^1$ , the set  $\mathcal{Y}$  is an  $\frac{1}{e^{\varepsilon} + 1}$ -target with  $(\varepsilon, \delta)$  for  $(\mathcal{M}(D^0), \mathcal{M}(D^1))$  (as in Definition B.1).
Consider two neighboring datasets. We have  $\mathcal{M}(D^0) \approx_{\varepsilon, \delta} \mathcal{M}(D^1)$ . Using Lemma A.1, for  $b \in \{0, 1\}$  we can have
$$
\mathcal {M} (D ^ {b}) = (1 - \delta) \cdot \mathbf {N} ^ {b} + \delta \cdot \mathbf {E} ^ {b}, \tag {2}
$$
where  $\mathbf{N}^0\approx_{\varepsilon}\mathbf{N}^1$ . From Lemma C.2,  $\mathcal{V}$  is a  $\frac{1}{e^{\varepsilon} + 1}$ -target with  $(\varepsilon ,0)$  for  $(\mathbf{N}^0,\mathbf{N}^1)$ . From Definition B.1 and (2), this means that  $\mathcal{V}$  is a  $\frac{1}{e^{\varepsilon} + 1}$ -target with  $(\varepsilon ,\delta)$  for  $(\mathcal{M}(D^0),\mathcal{M}(D^1))$ .
C.1 Proof of Lemma C.2
We first prove Lemma C.2 for the special case of private testing (when the support is  $\{0,1\}$ ):
Lemma C.3 (target for private testing). Let  $\mathbf{Z}^0$  and  $\mathbf{Z}^1$  with support  $\{0,1\}$  satisfy  $\mathbf{Z}^0 \approx_{\varepsilon} \mathbf{Z}^1$  Then  $\top = \{1\}$  (or  $\top = \{0\}$ ) is an  $\frac{1}{e^{\varepsilon} + 1}$ -target with  $(\varepsilon, 0)$  for  $(\mathbf{Z}^0, \mathbf{Z}^1)$ .
Proof. We show that Definition B.1 is satisfied with  $\top = \{1\}$ ,  $q = \frac{1}{e^{\varepsilon} + 1}$  and  $(\varepsilon, 0)$ , and  $\mathbf{Z}^0, \mathbf{Z}^1$ .
$$
\pi = \Pr [ \mathbf {Z} ^ {0} \in \mathsf {T} ]
$$
$$
\pi^ {\prime} = \operatorname * {P r} [ \mathbf {Z} ^ {1} \in \mathsf {T} ]
$$
be the probabilities of  $\top$  outcome in  $\mathbf{Z}^0$  and  $\mathbf{Z}^1$  respectively. Assume without loss of generality (otherwise we switch the roles of  $\mathbf{Z}^0$  and  $\mathbf{Z}^1$ ) that  $\pi' \geq \pi$ . If  $\pi \geq \frac{1}{e^{\varepsilon} + 1}$ , the choice of  $p = 0$  and  $\mathbf{B}^b = \mathbf{Z}^b$  (and any  $\mathbf{C}$ ) trivially satisfies the conditions of Definition 2.1. Generally, (also for all  $\pi < \frac{1}{e^{\varepsilon} + 1}$ ):
$$
p = 1 - \frac {\pi^ {\prime} e ^ {\varepsilon} - \pi}{e ^ {\varepsilon} - 1}.
$$
Note that since  $\mathbf{Z}^0\approx_{\varepsilon}\mathbf{Z}^1$  it follows that  $\pi^{\prime}\approx_{\varepsilon}\pi$  and  $(1 - \pi^{\prime})\approx_{\varepsilon}(1 - \pi)$  and therefore  $p\in [0,1]$  for any applicable  $0\leq \pi \leq \pi^{\prime}\leq 1$
- Let  $\mathbf{C}$  be the distribution with point mass on  $\bot = \{0\}$ .
Let  $\mathbf{B}^0 = \mathbf{Ber}\left(1 - \frac{\pi' - \pi}{\pi' - e^{-\varepsilon}\pi}\right) = \mathbf{Ber}\left(\frac{\pi - \pi e^{-\varepsilon}}{\pi' - e^{-\varepsilon}\pi}\right)$
- Let  $\mathbf{B}^1 = \mathbf{Ber}\left(1 - \frac{\pi' - \pi}{e^\varepsilon\pi' - \pi}\right) = \mathbf{Ber}\left(\frac{e^\varepsilon\pi' - \pi'}{e^\varepsilon\pi' - \pi}\right)$ .
We show that this choice satisfies Definition 2.1 with  $q = \frac{1}{e^{\varepsilon} + 1}$ .
- We show that for both  $b \in \{0,1\}$ .  $\mathbf{Z}^b \equiv p \cdot \mathbf{C} + (1 - p) \cdot \mathbf{B}^b$ : It suffices to show that the probability of  $\bot$  is the same for the distributions on both sides. For  $b = 0$ , the probability of  $\bot$  in the right hand side distribution is
$$
p + (1 - p) \cdot \frac {\pi^ {\prime} - \pi}{\pi^ {\prime} - e ^ {- \varepsilon} \pi} = 1 - \frac {\pi^ {\prime} e ^ {\varepsilon} - \pi}{e ^ {\varepsilon} - 1} + \frac {\pi^ {\prime} e ^ {\varepsilon} - \pi}{e ^ {\varepsilon} - 1} \cdot \frac {\pi^ {\prime} - \pi}{\pi^ {\prime} - e ^ {- \varepsilon} \pi} = 1 - \pi .
$$
For  $b = 1$ , the probability is
$$
\begin{array}{l} p + (1 - p) \cdot \frac {\pi^ {\prime} - \pi}{e ^ {\varepsilon} \pi^ {\prime} - \pi} = 1 - \frac {\pi^ {\prime} e ^ {\varepsilon} - \pi}{e ^ {\varepsilon} - 1} + \frac {\pi^ {\prime} e ^ {\varepsilon} - \pi}{e ^ {\varepsilon} - 1} \cdot \frac {\pi^ {\prime} - \pi}{e ^ {\varepsilon} \pi^ {\prime} - \pi} \\ = 1 - \frac {\pi^ {\prime} e ^ {\varepsilon} - \pi}{e ^ {\varepsilon} - 1} \left(1 - \frac {\pi^ {\prime} - \pi}{e ^ {\varepsilon} \pi^ {\prime} - \pi}\right) \\ = 1 - \frac {\pi^ {\prime} e ^ {\varepsilon} - \pi}{e ^ {\varepsilon} - 1} \cdot \frac {e ^ {\varepsilon} \pi^ {\prime} - \pi - \pi^ {\prime} + \pi}{e ^ {\varepsilon} \pi^ {\prime} - \pi} = 1 - \pi^ {\prime}. \\ \end{array}
$$
- We show that for  $b \in \{0,1\}$ ,  $\operatorname*{Pr}[\mathbf{B}^b \in \top] \geq \frac{1}{e^\varepsilon + 1}$ .
$$
\begin{array}{l} \operatorname * {P r} [ \mathbf {B} ^ {0} \in \mathsf {T} ] = \frac {\pi - e ^ {- \varepsilon} \pi}{\pi^ {\prime} - e ^ {- \varepsilon} \pi} \\ \geq \frac {\pi - e ^ {- \varepsilon} \pi}{e ^ {\varepsilon} \pi - e ^ {- \varepsilon} \pi} = \frac {e ^ {\varepsilon} - 1}{e ^ {2 \varepsilon} - 1} = \frac {1}{e ^ {\varepsilon} + 1}. \\ \end{array}
$$
$$
\begin{array}{l} \Pr [ \mathbf {B} ^ {1} \in \top ] = \frac {\pi^ {\prime} (e ^ {\varepsilon} - 1)}{\pi^ {\prime} e ^ {\varepsilon} - \pi} \\ \geq \frac {\pi (e ^ {\varepsilon} - 1)}{\pi e ^ {2 \varepsilon} - \pi} = \frac {e ^ {\varepsilon} - 1}{e ^ {2 \varepsilon} - 1} = \frac {1}{e ^ {\varepsilon} + 1} \\ \end{array}
$$
Note that the inequalities are tight when  $\pi' = \pi$  (and are tighter when  $\pi'$  is closer to  $\pi$ ). This means that our selected  $q$  is the largest possible that satisfies the conditions for the target being  $\top$ .
- We show that  $\mathbf{B}^0$  and  $\mathbf{B}^1$  are  $\varepsilon$ -indistinguishable, that is
$$
\mathbf {B e r} \left(1 - \frac {\pi^ {\prime} - \pi}{\pi^ {\prime} - e ^ {- \varepsilon} \pi}\right) \approx_ {\varepsilon} \mathbf {B e r} \left(1 - \frac {\pi^ {\prime} - \pi}{e ^ {\varepsilon} \pi^ {\prime} - \pi}\right).
$$
Recall that  $\mathbf{Ber}(a)\approx_{\varepsilon}\mathbf{Ber}(b)$  if and only if  $a\approx_{\varepsilon}b$  and  $(1 - a)\approx_{\varepsilon}(1 - b)$ . First note that
$$
e ^ {- \varepsilon} \cdot \frac {\pi^ {\prime} - \pi}{\pi^ {\prime} - e ^ {- \varepsilon} \pi} = \frac {\pi^ {\prime} - \pi}{e ^ {\varepsilon} \pi^ {\prime} - \pi}
$$
$$
\frac {\pi^ {\prime} - \pi}{\pi^ {\prime} - e ^ {- \varepsilon} \pi} \approx_ {\varepsilon} \frac {\pi^ {\prime} - \pi}{e ^ {\varepsilon} \pi^ {\prime} - \pi}.
$$
It also holds that
$$
1 \leq \frac {\frac {\pi - e ^ {- \varepsilon} \pi}{\pi^ {\prime} - e ^ {- \varepsilon} \pi}}{\frac {\pi^ {\prime} (1 - e ^ {- \varepsilon})}{\pi^ {\prime} - e ^ {- \varepsilon} \pi}} = \frac {\pi^ {\prime}}{\pi} \leq e ^ {\varepsilon}.
$$
Proof of Lemma C.2. The proof is very similar to that of Lemma C.3, with a few additional details since  $\top = \mathcal{Y}$  can have more than one element (recall that  $\bot$  is a single element).
Assume (otherwise we switch roles) that  $\operatorname{Pr}[\mathbf{Z}^0 = \bot] \geq \operatorname{Pr}[\mathbf{Z}^1 = \bot]$ . Let
$$
\pi = \Pr [ \mathbf {Z} ^ {\mathbf {0}} \in \mathcal {Y} ]
$$
$$
\pi^ {\prime} = \Pr [ \mathbf {Z} ^ {1} \in \mathcal {Y} ].
$$
Note that  $\pi' \geq \pi$ .
We choose  $p$ ,  $\mathbf{C}$ ,  $\mathbf{B}^0$ ,  $\mathbf{B}^1$  as follows. Note that when  $\pi \geq \frac{1}{e^\varepsilon + 1}$ , then the choice of  $p = 0$  and  $\mathbf{B}^b = \mathbf{Z}^b$  satisfies the conditions. Generally,
- Let  $\mathbf{C}$  be the distribution with point mass on  $\perp$ .
- Let  $\mathbf{B}^0$  be  $\perp$  with probability  $\frac{\pi' - \pi}{\pi' - e^{-\varepsilon}\pi}$  and otherwise (with probability  $\frac{\pi - \pi e^{-\varepsilon}}{\pi' - e^{-\varepsilon}\pi}$ ) be  $\mathbf{Z}^0$  conditioned on the outcome being in  $\mathcal{Y}$ .
- Let  $\mathbf{B}^1$  be  $\perp$  with probability  $\frac{\pi' - \pi}{e^{\varepsilon}\pi' - \pi}$  and otherwise (with probability  $\frac{e^{\varepsilon}\pi' - \pi'}{e^{\varepsilon}\pi' - \pi}$ ) be  $\mathbf{Z}^1$  conditioned on the outcome being in  $\mathcal{V}$ .
It remains to show that these choices satisfy Definition 2.1:
The argument for  $\operatorname*{Pr}[\mathbf{B}^b\in \mathcal{Y}]\geq \frac{e^\varepsilon - 1}{e^{2\varepsilon} - 1}$  is identical to Lemma C.3 (with  $\mathcal{V} = \top$ ).
We next verify that for  $b \in \{0,1\}$ :  $\mathbf{Z}^b \equiv p \cdot \mathbf{C} + (1 - p) \cdot \mathbf{B}^b$ . The argument for the probability of  $\bot$  is identical to Lemma C.3. The argument for  $y \in \mathcal{V}$  follows from the probability of being in  $\mathcal{V}$  being the same and that proportions are maintained.
For  $b = 0$ , the probability of  $y \in \mathcal{V}$  in the right hand side distribution is
$$
(1 - p) \cdot \frac {\pi - \pi e ^ {- \varepsilon}}{\pi^ {\prime} - e ^ {- \varepsilon} \pi} \cdot \frac {\Pr [ \mathbf {Z} ^ {0} = y ]}{\Pr [ \mathbf {Z} ^ {0} \in \mathcal {Y} ]} = \pi \cdot \frac {\Pr [ \mathbf {Z} ^ {0} = y ]}{\Pr [ \mathbf {Z} ^ {0} \in \mathcal {Y} ]} = \Pr [ \mathbf {Z} ^ {0} = y ].
$$
For  $b = 1$ , the probability of  $y \in \mathcal{V}$  in the right hand side distribution is
$$
\begin{array}{l} (1 - p) \cdot \frac {e ^ {\varepsilon} \pi^ {\prime} - \pi^ {\prime}}{e ^ {\varepsilon} \pi^ {\prime} - \pi} \cdot \frac {\mathrm {P r} [ \mathbf {Z} ^ {1} = y ]}{\mathrm {P r} [ \mathbf {Z} ^ {1} \in \mathcal {Y} ]} = \pi^ {\prime} \cdot \frac {\mathrm {P r} [ \mathbf {Z} ^ {1} = y ]}{\mathrm {P r} [ \mathbf {Z} ^ {1} \in \mathcal {Y} ]} \\ = \Pr [ \mathbf {Z} ^ {1} = y ]. \\ \end{array}
$$
Finally, we verify that  $\mathbf{B}^0$  and  $\mathbf{B}^1$  are  $\varepsilon$ -indistinguishable. Let  $W \subset \mathcal{V}$ . We have
$$
\Pr [ \mathbf {B} ^ {0} \in W ] = \frac {\pi (1 - e ^ {- \varepsilon})}{\pi^ {\prime} - e ^ {- \varepsilon} \pi} \cdot \frac {\Pr [ \mathbf {Z} ^ {0} \in W ]}{\pi} = \frac {e ^ {\varepsilon} - 1}{\pi^ {\prime} e ^ {\varepsilon} - \pi} \Pr [ \mathbf {Z} ^ {0} \in W ]
$$
$$
\operatorname * {P r} [ \mathbf {B} ^ {1} \in W ] = \frac {\pi^ {\prime} (e ^ {\varepsilon} - 1)}{e ^ {\varepsilon} \pi^ {\prime} - \pi} \cdot \frac {\operatorname * {P r} [ \mathbf {Z} ^ {1} \in W ]}{\pi^ {\prime}} = \frac {e ^ {\varepsilon} - 1}{\pi^ {\prime} e ^ {\varepsilon} - \pi} \operatorname * {P r} [ \mathbf {Z} ^ {1} \in W ].
$$
$$
\frac {\Pr [ \mathbf {B} ^ {0} \in W ]}{\Pr [ \mathbf {B} ^ {1} \in W ]} = \frac {\Pr [ \mathbf {Z} ^ {0} \in W ]}{\Pr [ \mathbf {Z} ^ {1} \in W ]}
$$
and we use  $\mathbf{Z}^0\approx_{\varepsilon}\mathbf{Z}^1$ . The case of  $W = \perp$  is identical to the proof of Lemma C.3. The case  $\perp \in W$  follows.
D Conditional Release with Revisions
In this section we analyze an extension to conditional release that allows for revision calls to be made with respect to previous computations. This extension was presented in Section 2.3 and described in Algorithm 2. A conditional release applies a private algorithm  $\mathcal{A} \to \mathcal{Y}$  with respect to a subset of outcomes  $\top \subset \mathcal{Y}$ . It draws  $y \sim \mathcal{A}(D)$  and returns  $y$  if  $y \in \top$  and  $\bot$  otherwise. Each revise call effectively expands the target to  $\top_h \cup \top'$ , when  $\top_h$  is the prior target and  $\top'$  a disjoint extension. If the (previously) computed result hits the expanded target ( $y \in \top'$ ), the value  $y$  is reported and charged. Otherwise, additional revise calls can be performed. The revise calls can be interleaved with other TCT computations at any point in the interaction.
D.1 Preliminaries
For a distribution  $\mathbf{Z}$  with support  $\mathcal{V}$  and  $W\subset \mathcal{V}$  we denote by  $\mathbf{Z}_W$  the distribution with support  $W\cup \{\bot \}$  where outcomes not in  $W$  are "replaced" by  $\perp$ . That is, for  $y\in W$ ,  $\operatorname*{Pr}[\mathbf{Z}_W = y]\coloneqq \operatorname*{Pr}[\mathbf{Z} = y]$  and  $\operatorname*{Pr}[\mathbf{Z}_W = \perp ]\coloneqq \operatorname*{Pr}[\mathbf{Z}\notin W]$ .
For a distribution  $\mathbf{Z}$  with support  $\mathcal{V}$  and  $W\subset \mathcal{V}$  we denote by  $\mathbf{Z}\mid W$  the conditional distribution of  $\mathbf{Z}$  on  $W$ . That is, for  $y\in W$ ,  $\operatorname*{Pr}[(\mathbf{Z}\mid W) = y]\coloneqq \operatorname*{Pr}[\mathbf{Z} = y] / \operatorname*{Pr}[\mathbf{Z}\in W]$ .
Lemma D.1. If  $\mathbf{B}^0\approx_{\varepsilon ,\delta}\mathbf{B}^1$  then  $\mathbf{B}_W^0\approx_{\varepsilon ,\delta}\mathbf{B}_W^1$
Lemma D.2. Let  $\mathbf{B}^0$ ,  $\mathbf{B}^1$  be probability distributions with support  $\mathcal{V}$  such that  $\mathbf{B}^0 \approx_{\varepsilon} \mathbf{B}^1$ . Let  $W \subset \mathcal{V}$ . Then  $\mathbf{B}^0 \mid W \approx_{2\varepsilon} \mathbf{B}^1 \mid W$ .
We extend these definitions to a randomized algorithm  $\mathcal{A}$ , where  $\mathcal{A}_W(D)$  has distribution  $\mathcal{A}(D)_W$  and  $(\mathcal{A} \mid W)(D)$  has distribution  $\mathcal{A}(D) \mid W$ . The claims in Lemma D.1 and Lemma D.2 then transfer to privacy of the algorithms.
D.2 Analysis
To establish correctness, it remains to show that each ConditionalRelease call with an  $(\varepsilon, \delta)$ -DP algorithm  $\mathcal{A}$  can be casted in TCT as a call to an  $(\varepsilon, \delta)$ -DP algorithm with a NotPrior target and each ReviseCR call cap can be casted as a call to an  $2\varepsilon$ -DP algorithm with a NotPrior target.
Proof of Lemma 2.5. The claim for ConditionalRelease was established in Lemma 2.4: Conditional release ConditionalRelease  $(\mathcal{A},\top)$  calls the algorithm  $\mathcal{A}_{\top}$  with target  $\top$ . From Lemma D.1,  $\mathcal{A}_{\top}$  is  $(\varepsilon ,\delta)$ -DP when  $\mathcal{A}$  is  $(\varepsilon ,\delta)$ -DP.  $\top$  constitutes a NotPrior target for  $\mathcal{A}_{\top}$  with respect to prior  $\bot$ .
We next consider revision calls as described in Algorithm 2. We first consider the case of a pure-DP  $\mathcal{A}(\delta = 0)$ .
When ConditionalRelease publishes  $\perp$ , the internally stored value  $r_h$  conditioned on published  $\perp$  is a sample from the conditional distribution  $\mathcal{A}(D) \mid \neg \top$ .
We will show by induction that this remains true after ReviseCR calls, that is the distribution of  $r_h$  conditioned on  $\bot$  being returned in all previous calls is  $\mathcal{A}(D) \mid \neg \top_h$  where  $\top_h$  is the current expanded target.
An ReviseCR call with respect to current target  $\top_h$  and extension  $\top'$  can be equivalently framed as drawing  $r \sim \mathcal{A}(D) \mid \neg \top_h$ . From Lemma D.2, if  $\mathcal{A}$  is  $\varepsilon$ -DP then  $\mathcal{A} \mid \neg \top_h$  is  $2\varepsilon$ -DP. If  $r \in \top'$  we publish it and otherwise we publish  $\bot$ . This is a conditional release computation with respect to the  $2\varepsilon$ -DP algorithm  $\mathcal{A} \mid \neg \top_h$  and the target  $\top'$ . Equivalently, it is a call to the  $2\varepsilon$ -DP algorithm  $(\mathcal{A} \mid \neg \top_h)_{\top'}$  with a NotPrior target  $\top'$ .
Following the ReviseCR call, the conditional distribution of  $r_h$  conditioned on  $\bot$  returned in the previous calls is  $\mathcal{A}(D) \mid \neg (\mathsf{T}_h \cup \mathsf{T}'')$  as claimed. We then update  $\mathsf{T}_h \gets \mathsf{T}_h \cup \mathsf{T}'$ .
It remains to handle the case  $\delta > 0$ . We consider ReviseCR calls for the case where  $\mathcal{A}$  is  $(\varepsilon, \delta)$ -DP (approximate DP). In this case, we want to show that we charge for the  $\delta$  value once, only on the original ConditionalRelease call. We apply the simulation-based analysis in the proof of Theorem B.4 with two fixed neighboring datasets. Note that this can be viewed as each call being
with a pair of distributions with an appropriate  $q$ -target (that in our case is always a NotPrior target).
The first ConditionalRelease call uses the distributions  $\mathcal{A}(D^0)$  and  $\mathcal{A}(D^1)$ . From Lemma A.1 they can be expressed as respective mixtures of pure  $\mathbf{N}^0 \approx_{\varepsilon} \mathbf{N}^1$  part (with probability  $1 - \delta$ ) and non-private parts. The non-private draw is designated failure with probability  $\delta$ . Effectively, the call in the simulation is then applied to the pair  $(\mathbf{N}_\top^0, \mathbf{N}_\top^1)$  with target  $\top$ .
A followup ReviseCR call is with respect to the previous target  $\top_h$  and target extension  $\top'$ . The call is with the distributions  $(\mathbf{N}^b \mid \neg \top_h)_{\top'}$  that using Lemma D.1 and Lemma D.2 satisfy  $(\mathbf{N}^0 \mid \neg \top_h)_{\top'} \approx_{2\varepsilon} (\mathbf{N}^1 \mid \neg \top_h)_{\top'}$ .
E Boundary Wrapper Analysis
In this section we provide details for the boundary wrapper method including proofs of Lemma 2.7 and Lemma 2.8. For instructive reasons, we first consider the special case of private testing and then outline the extensions to private classification.
Algorithm 4 when specialized for tests first computes  $\pi(D) = \min\{\operatorname*{Pr}[\mathcal{A}(D) = 0], 1 - \operatorname*{Pr}[\mathcal{A}(D) = 0]\}$ , returns  $\top$  with probability  $\pi/(1+\pi)$  and otherwise (with probability  $1/(1+\pi)$ ) return  $\mathcal{A}(D)$ . Overall, we return the less likely outcome with probability  $\pi/(1+\pi)$ , and the more likely one with probability  $(1-\pi)/(1+\pi)$ .
Lemma E.1 (Privacy of wrapped test). If the test is  $\varepsilon$ -DP then the wrapper test is  $t(\varepsilon)$ -DP where  $t(\varepsilon) \leq \frac{4}{3}\varepsilon$ .
Proof. Working directly with the definitions,  $t(\varepsilon)$  is the maximum of
$$
\left. \max  _ {\pi \in (0, 1 / 2)} \left| \ln \left(\frac {1 - e ^ {- \varepsilon} \pi}{1 + e ^ {- \varepsilon} \pi} \cdot \frac {1 + \pi}{1 - \pi}\right) \right| \leq \frac {4}{3} \varepsilon \right. \tag {3}
$$
$$
\left. \max  _ {\pi \in (0, 1 / 2)} \left| \ln \left(\frac {e ^ {- \varepsilon} \pi}{1 + e ^ {- \varepsilon} \pi} \cdot \frac {1 + \pi}{\pi}\right) \right| \leq \varepsilon \right. \tag {4}
$$
$$
\max  _ {\pi \in \left(\frac {e ^ {- \varepsilon}}{2}, \frac {1}{1 + e ^ {\varepsilon}}\right)} \left| \ln \left(\frac {\pi}{1 + \pi} \cdot \frac {2 - e ^ {\varepsilon} \pi}{e ^ {\varepsilon} \pi}\right) \right| \leq \varepsilon \tag {5}
$$
$$
\max  _ {\pi \in \left(\frac {e ^ {- \varepsilon}}{2}, \frac {1}{1 + e ^ {\varepsilon}}\right)} \left| \ln \left(\frac {1 - \pi}{1 + \pi} \cdot \frac {2 - e ^ {\varepsilon} \pi}{1 - e ^ {\varepsilon} \pi}\right) \right| \leq \frac {4}{3} \varepsilon \tag {6}
$$
$$
\left. \max  _ {\pi \in \left(\frac {e ^ {- \varepsilon}}{2}, \frac {1}{1 + e ^ {\varepsilon}}\right)} \left| \ln \left(\frac {\pi}{1 + \pi} \cdot \frac {2 - e ^ {\varepsilon} \pi}{1 - e ^ {\varepsilon} \pi}\right) \right| \leq \varepsilon \right. \tag {7}
$$
Inequality (3) bounds the ratio change in the probably of the larger probability outcome when it remains the same and (4) the ratio change in the probability of the smaller probability outcome when it remains the same between the neighboring datasets. When the less probable outcome changes between the neighboring datasets it suffices to consider the case where the probability of the initially less likely outcome changes to  $e^{\varepsilon}\pi > 1/2$  so that  $e^{\varepsilon}\pi < 1 - \pi$ , that is the change is from  $\pi$  to  $e^{\varepsilon}\pi$  where  $\pi \in \left(\frac{e^{-\varepsilon}}{2}, \frac{1}{1 + e^{\varepsilon}}\right)$ . Inequalities 5 and 6 correspond to this case. The wrapped probabilities of the  $\top$  outcome are the same as the less probably outcome in the case that it is the same in the two databases. Inequality 7 corresponds to the case when there is change.
We now show that  $\top$  is a target for the wrapped test.
Lemma E.2 (q-value of the boundary target). The outcome  $\top$  of a boundary wrapper of an  $\varepsilon$ -DP test is a  $\frac{e^{t(\varepsilon)} - 1}{2(e^{\varepsilon + t(\varepsilon)} - 1)}$ -target.
Proof. Consider two neighboring datasets where the same outcome is less likely for both and  $\pi \leq \pi'$ . Suppose without loss of generality that 0 is the less likely outcome.
The common distribution  $(C)$  has point mass on 1.
The distribution  $\mathbf{B}^0$  is a scaled part of  $\mathcal{M}(D^0)$  that includes all 0 and  $\top$  outcomes (probability  $\pi / (1 + \pi)$  each) and probability of  $\Delta \frac{e^{t(\varepsilon)}}{e^{t(\varepsilon)} - 1}$  of the 1 outcomes, where  $\Delta = \frac{2\pi'}{1 + \pi'} - \frac{2\pi}{1 + \pi}$ .
The distribution  $\mathbf{B}^1$  is a scaled part of  $\mathcal{M}(D^1)$  that includes all 0 and  $\top$  outcomes (probability  $\pi' / (1 + \pi')$  each) and probability of  $\Delta \frac{1}{e^{t(\varepsilon)} - 1}$  of the 1 outcomes.
It is easy to verify that  $\mathbf{B}^0\approx_{t(\varepsilon)}\mathbf{B}^1$  and that
$$
\begin{array}{l} 1 - p = \frac {2 \pi^ {\prime}}{1 + \pi^ {\prime}} + \Delta \frac {1}{e ^ {t (\varepsilon)} - 1} = \frac {2 \pi}{1 + \pi} + \Delta \frac {e ^ {t (\varepsilon)}}{e ^ {t (\varepsilon)} - 1} \\ = \frac {2 \pi^ {\prime}}{1 + \pi^ {\prime}} \frac {e ^ {t (\varepsilon)}}{e ^ {t (\varepsilon)} - 1} - \frac {2 \pi}{1 + \pi} \frac {1}{e ^ {t (\varepsilon)} - 1} \\ = \frac {2}{e ^ {t (\varepsilon)} - 1} \left(e ^ {t (\varepsilon)} \frac {\pi^ {\prime}}{1 + \pi^ {\prime}} - \frac {\pi}{1 + \pi}\right) \\ \end{array}
$$
Using  $\frac{\pi}{1 + \pi} \leq \frac{\pi'}{1 + \pi'}$  and  $\frac{\frac{\pi'}{1 + \pi'}}{\frac{\pi}{1 + \pi}} \leq e^{\varepsilon}$  we obtain
$$
\begin{array}{l} q \geq \frac {\frac {\pi}{1 + \pi}}{1 - p} \\ = \frac {e ^ {t (\varepsilon)} - 1}{2} \left(\frac {1}{e ^ {t (\varepsilon)} \cdot \frac {\pi^ {\prime}}{1 + \pi^ {\prime}} \cdot \frac {1 + \pi}{\pi} - 1}\right) \\ \geq \frac {e ^ {t (\varepsilon)} - 1}{2} \frac {1}{e ^ {t (\varepsilon) + \varepsilon} - 1}. \\ \end{array}
$$