SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Extension to Private Classification To extension from Lemma E.1 to Lemma 2.7 follows by noting that the same arguments also hold respectively for sets of outcomes and also cover the case when there is no dominant outcome and when there is a transition between neighboring datasets from no dominant outcome to a dominant outcome. The extension from Lemma E.1 to Lemma 2.7 is also straightforward by also noting the cases above (that only make the respective  $\Delta$  smaller), and allowing C to be empty when there is no dominant outcome.
F Boundary wrapping without a probability oracle
We present a boundary-wrapping method that does not assume a probability oracle. This method accesses the distribution  $\mathcal{A}(D)$  in a blackbox fashion.
At a very high level, we show that one can run an  $(\varepsilon, 0)$ -DP algorithm  $\mathcal{A}$  twice and observe both outcomes. Then, denote by  $\mathcal{V}$  the range of the algorithm  $\mathcal{A}$ . We can show that  $E = \{(y, y') : y \neq y'\} \subseteq \mathcal{V} \times \mathcal{V}$  is an  $\Omega(1)$ -target of this procedure. That is, if the analyst observes the same outcome twice, she learns the outcome "for free". If the two outcomes are different, the analyst pays  $O(\varepsilon)$  of privacy budget, but she will be able to access both outcomes, which is potentially more informative than a single execution of the algorithm.
Lemma F.1. Suppose  $\mathcal{A}:\mathcal{X}^*\to \mathcal{Y}$  is an  $(\varepsilon ,0)$ -DP algorithm where  $|\mathcal{Y}| < \infty$ . Denote by  $\mathcal{A}\circ \mathcal{A}$  the following algorithm: on input  $D$ , independently run  $\mathcal{A}$  twice and publish both outcomes. Define  $E\coloneqq \{(y,y'):y\neq y'\} \subseteq \mathcal{Y}\times \mathcal{Y}$ . Then,  $\mathcal{A}\circ \mathcal{A}$  is a  $(2\varepsilon ,0)$ -DP algorithm, and  $E$  is a  $f(\varepsilon)$ -target for  $\mathcal{A}\circ \mathcal{A}$ , where
$$
f (\varepsilon) = 1 - \sqrt {e ^ {2 \varepsilon} / (1 + e ^ {2 \varepsilon})}.
$$
Proof.  $\mathcal{A} \circ \mathcal{A}$  is  $(2\varepsilon, 0)$ -DP by the basic composition theorem. Next, we verify the second claim.
Identify elements of  $\mathcal{V}$  as  $1,2,\ldots,m = |\mathcal{V}|$ . Let  $D,D^{\prime}$  be two adjacent data sets. For each  $i\in [m]$ , let
$$
p _ {i} = \Pr [ \mathcal {A} (D) = i ], \quad p _ {i} ^ {\prime} = \Pr [ \mathcal {A} (D ^ {\prime}) = i ].
$$
We define a distribution  $\mathbf{C}$ . For each  $i\in [m]$ , define  $q_{i}$  to be the largest real such that
$$
p _ {i} ^ {2} - q _ {i} \in [ e ^ {- 2 \varepsilon} (p _ {i} ^ {\prime 2} - q _ {i}), e ^ {2 \varepsilon} (p _ {i} ^ {\prime 2} - q _ {i}) ].
$$
Then, we define  $\mathbf{C}$  to be a distribution over  $\{(i,i):i\in [m]\}$  where  $\operatorname *{Pr}[\mathbf{C} = (i,i)] = \frac{q_i}{\sum_jq_j}$ .
We can then write  $(\mathcal{A} \circ \mathcal{A})(D) = \alpha \cdot \mathbf{C} + (1 - \alpha) \cdot \mathbf{N}^0$  and  $(\mathcal{A} \circ \mathcal{A})(D') = \alpha \cdot \mathbf{C} + (1 - \alpha) \cdot \mathbf{N}^1$ , where  $\alpha = \sum_{i} q_i$ , and  $\mathbf{N}^0$  and  $\mathbf{N}^1$  are  $2\varepsilon$ -indistinguishable.
Next, we consider lower-bounding  $\operatorname*{Pr}[\mathbf{N}^0 = (y,y'):y\neq y']$  . The lower bound of  $\operatorname*{Pr}[\mathbf{N}^0 = (y,y'):y\neq y']$  will follow from the same argument.
Indeed, we have
$$
\frac {\operatorname * {P r} [ \mathbf {N} ^ {0} = (y , y ^ {\prime}) : y \neq y ^ {\prime} ]}{\operatorname * {P r} [ \mathbf {N} ^ {0} = (y , y) ]} = \frac {\sum_ {i} p _ {i} (1 - p _ {i})}{\sum_ {i} p _ {i} ^ {2} - q _ {i}}.
$$
$$
p _ {i} ^ {2} - q _ {i} \leq 1 - p _ {i} ^ {\prime 2}.
$$
The inequality is trivially true if  $p_i^2 \leq 1 - p_i'^2$ . Otherwise, we can observe that for  $q \coloneqq p_i^2 + p_i'^2 - 1 > 0$ , we have  $p_i^2 - q = 1 - p_i'^2$  and  $p_i'^2 - q = 1 - p_i^2$ . Since  $1 - p_i'^2 \in [e^{-2\varepsilon}(1 - p_i^2), e^{2\varepsilon}(1 - p_i^2)]$ , this implies that  $q_i$  can only be larger than  $q$ .
Since we also trivially have that  $p_i^2 - q_i \leq p_i^2$ , we conclude that
$$
\frac {\operatorname * {P r} [ \mathbf {N} ^ {0} = (y , y ^ {\prime}) : y \neq y ^ {\prime} ]}{\operatorname * {P r} [ \mathbf {N} ^ {0} = (y , y) ]} \geq \frac {\sum_ {i} p _ {i} (1 - p _ {i})}{\sum_ {i} \min (p _ {i} ^ {2} , 1 - p _ {i} ^ {\prime 2})} \geq \frac {\sum_ {i} p _ {i} (1 - p _ {i})}{\sum_ {i} \min (p _ {i} ^ {2} , e ^ {2 \varepsilon} (1 - p _ {i} ^ {2}))}.
$$
Next, it is straightforward to show that, for every  $p \in [0,1]$ , one has
$$
\frac {p (1 - p)}{\min (p ^ {2} , e ^ {2 \varepsilon} (1 - p ^ {2}))} = \min \left(\frac {1 - p}{p}, \frac {p}{e ^ {2 \varepsilon} (1 + p)}\right) \geq \frac {1 - \sqrt {e ^ {2 \varepsilon} / (1 + e ^ {2 \varepsilon})}}{\sqrt {e ^ {2 \varepsilon} / (1 + e ^ {2 \varepsilon})}}.
$$
$$
\operatorname * {P r} [ \mathbf {N} ^ {0} = (y, y ^ {\prime}): y \neq y ^ {\prime} ] = \frac {\operatorname * {P r} [ \mathbf {N} ^ {0} = (y , y ^ {\prime}) : y \neq y ^ {\prime} ]}{\operatorname * {P r} [ \mathbf {N} ^ {0} = (y , y ^ {\prime}) : y \neq y ^ {\prime} ] + \operatorname * {P r} [ \mathbf {N} ^ {0} = (y , y) ]} \geq 1 - \sqrt {e ^ {2 \varepsilon} / (1 + e ^ {2 \varepsilon})},
$$
Remark F.2. For a typical use case where  $\epsilon = 0.1$ , we have  $f(\varepsilon) \approx 0.258$ . Then, by applying Theorem B.4, on average we pay  $\approx 8\varepsilon$  privacy cost for each target hit. Improving the constant of 8 is a natural question for future research. We also note that while the overhead is more significant compared to the boundary wrapper of Algorithm 4, the output is more informative as it includes two independent responses of the core algorithm whereas Algorithm 4 returns one or none (when  $\top$  is returned). We expect that it is possible to design less-informative boundary wrappers for the case of blackbox access (no probability oracle) that have a lower overhead. We leave this as an interestion question for followup work.
G  $q$  value for BetweenThresholds
We provide details for the BetweenThresholds classifier (see Section 2.6). The BetweenThresholds classifier is a refinement of AboveThreshold. It is specified by a 1-Lipschitz function  $f$ , two thresholds  $t_\ell < t_r$ , and a privacy parameter  $\varepsilon$ . We compute  $\tilde{f}(D) = f(D) + \mathbf{Lap}(1 / \varepsilon)$ , where  $\mathbf{Lap}$  is the Laplace distribution. If  $\tilde{f}(D) < t_\ell$  we return L. If  $\tilde{f}(D) > t_r$  we return H. Otherwise, we return T.
Lemma G.1 (Effectiveness of the "between" target). The  $\top$  outcome is an  $(1 - e^{-(t_r - t_l)\varepsilon})\cdot \frac{e^{\varepsilon} - 1}{e^{2\varepsilon} - 1}$  target for BetweenThresholds.
Proof. Without loss of generality we assume that  $t_\ell = 0$  and  $t_r = t / \varepsilon$  for a parameter  $t$ .
Consider two neighboring data sets  $D^0$  and  $D^1$  and the respective  $f(D^0)$  and  $f(D^1)$ . Since  $f$  is 1-Lipschitz, we can assume without loss of generality (otherwise we switch the roles of the two data
sets) that  $f(D^0) \leq f(D^1) \leq f(D^0) + 1$ . Consider the case  $f(D^1) \leq 0$ . The case  $f(D^0) \geq t / \varepsilon$  is symmetric and the cases where one or both of  $f(D^b)$  are in  $(0, t / \varepsilon)$  make  $\bot$  a more effective target.
$$
\pi_ {L} ^ {b} := \Pr [ f (D ^ {b}) + \mathbf {L a p} (1 / \varepsilon) <   t _ {\ell} = 0 ] = 1 - \frac {1}{2} e ^ {- | f (D ^ {b}) | \varepsilon}
$$
$$
\pi_ {H} ^ {b} := \Pr [ f (D ^ {b}) + \mathbf {L a p} (1 / \varepsilon) > t _ {r} = t / \varepsilon ] = \frac {1}{2} e ^ {- (| f (D ^ {b}) | \varepsilon - t}
$$
$$
\pi_ {\top} ^ {b} := \Pr [ f (D ^ {b}) + \mathbf {L a p} (1 / \varepsilon) \in (0, t / \varepsilon) ] = \frac {1}{2} \left(e ^ {- | f (D ^ {b}) | \varepsilon} - e ^ {- (| f (D ^ {b}) | \varepsilon - t)}\right) = \frac {1}{2} e ^ {- | f (D ^ {b}) | \varepsilon} (1 - e ^ {- t})
$$
Note that  $\pi_L^0\approx_{\varepsilon}\pi_L^1$  and  $\pi_H^1\approx_{\varepsilon}\pi_H^0,\pi_L^0\geq \pi_L^1$  and  $\pi_H^1\geq \pi_H^0$
$$
p = (\pi_ {L} ^ {1} - \frac {1}{e ^ {\varepsilon} - 1} (\pi_ {L} ^ {0} - \pi_ {L} ^ {1})) + (\pi_ {H} ^ {0} - \frac {1}{e ^ {\varepsilon} - 1} (\pi_ {H} ^ {1} - \pi_ {H} ^ {0}))
$$
and the distribution  $\mathbf{C}$  to be L with probability  $(\pi_L^1 -\frac{1}{e^\varepsilon - 1} (\pi_L^0 -\pi_L^1)) / p$  and H otherwise.
We specify  $p$  and the distributions  $\mathbf{B}^b$  and  $\mathbf{C}$  as we did for NotPrior (Lemma 2.3) with respect to "prior" L. (We can do that and cover also the case where  $f(D^0) > t / \varepsilon$  where the symmetric prior would be H because the target does not depend on the values being below or above the threshold).
The only difference is that our target is smaller, and includes only  $\top$  rather than  $\top$  and H. Because of that, the calculated  $q$  value is reduced by a factor of
$$
\frac {\pi_ {\top} ^ {b}}{\pi_ {\top} ^ {b} + \pi_ {H} ^ {b}} = \frac {\frac {1}{2} e ^ {- | f (D ^ {b}) | \varepsilon} (1 - e ^ {- t})}{\frac {1}{2} e ^ {- | f (D ^ {b}) | \varepsilon}} = (1 - e ^ {- t}).
$$
H Numerical Comparison of BetweenThresholds
We demonstrate numerically the benefits of our TCT-based BetweenThresholds (Section 2.6) compared with the SVT-based prior work of [3].
Consider a sensitive dataset  $D = \{x_{1},\ldots ,x_{n}\}$  of size  $n$  and thresholds  $t_l < t_r$ . We process a sequence of queries where each is specified by a function  $f:X\to [0,1]$ . We aim to process each query by performing a noisy comparison of  $\frac{1}{n}\sum_{i = 1}^{n}f(x_i)$  to  $t_l$  and  $t_r$  with the goal of having a error of at most  $\alpha$  on our responses to queries with confidence at least  $\beta = 1 - 1 / n$ . That is, for below responses it holds that  $\frac{1}{n}\sum_{i = 1}^{n}f(x_i)\leq t_l + \alpha$ , for above responses it holds that  $\frac{1}{n}\sum_{i = 1}^{n}f(x_i)\geq t_r - \alpha$ , and for between responses it holds that  $t_l - \alpha \leq \frac{1}{n}\sum_{i = 1}^{n}f(x_i)\leq t_r + \alpha$ .
We then consider how many  $\top$  (that is, between) responses we can incur under the requirement that the whole sequence is  $(\varepsilon, \delta)$ -DP with  $\varepsilon \leq 1$  and  $\delta = 1/n$ .
TCT: For a query  $f$ , we compare the noisy count  $\frac{1}{n} \sum_{i=1}^{n} f(x_i) + \mathbf{Lap}\left(\frac{\alpha}{\log(n)}\right)$  with the thresholds  $t_l, t_r$  and report below, between  $(\top)$ , or above accordingly. To achieve the desired accuracy and confidence of  $\alpha$  and  $\beta = 1 - 1/n$ , we set the Laplace noise parameter to  $\frac{\alpha}{\log(n)}$ . This ensures that with confidence at least  $(1 - \frac{1}{n})$ , the noisy count is within relative error  $\alpha$  of the true count. The sensitivity is  $1/n$  and each call is  $\varepsilon'$ -DP with  $\varepsilon' = \frac{\log n}{\alpha n}$ .
For  $t_r - t_l \geq \frac{2}{\varepsilon'}$ , each call has a  $q$ -target with  $q = \frac{1 - e^{-2}}{1 + e^{\varepsilon'}}$ . We can then compute the overall privacy cost for a number of target hits using basic or advanced composition according to Theorem B.4 using the appropriate  $q$  values and  $\delta$ . We solve for the number of target hits that keeps the overall privacy parameter values below  $(1, 1/n)$ .
SVT-based [3]: Through the analysis in [3], we obtain that we need  $\varepsilon' \geq \frac{16 \cdot \log(n)}{\alpha n}$ . An "optimistic" analysis that we performed implied that we can not get better than  $\varepsilon' \geq \frac{4 \cdot \log(n)}{\alpha n}$ .
Figure 1 shows the dependence of the number of between responses ( $y$  axis) on the number of data points  $n$  ( $x$  axis) for TCT and for the provided and optimistic bounds for [3]. TCT has a factor of 95 gain with respect to the provided bound and a factor of 6 gain over the optimistic bound.
Figure 1: Number of BetweenThresholds target hits for data set size.  $\delta = \beta = 1 / n, \alpha = 0.01$
Finally, the gains reported in the figure assume that both algorithms use the same gap that is large enough to satisfy the requirement of [3] that  $|t_r - t_l| > = \frac{12(\log(1 / \varepsilon') + \log(1 / \delta))}{\varepsilon'}$ . But in situations where we are interested in a narrow gap around a single threshold, TCT with the smaller effective gap also benefits from much fewer target hits.
I Analysis of SVT with individual privacy charging
Our improved SVT with individual charging is described in Algorithm 5 (see Section 2.8).
We establish the privacy guarantee:
Proof of Theorem 2.9. We apply simulation-based privacy analysis (see Section A.5). Consider two neighboring datasets  $D$  and  $D' = D \cup \{x\}$ . The only queries where potentially  $f(D) \neq f(D')$  and we may need to call the data holder are those with  $f(x) \neq 0$ . Note that for every  $x' \in D$ , the counter  $C_{x'}$  is the same during the execution of Algorithm 5 on either  $D$  or  $D'$ . This is because the update of  $C_{x'}$  depends only on the published results and  $f_i(x')$ , both of which are public information. Hence, we can think of the processing of  $C_{x'}$  as a post-processing when we analyze the privacy property between  $D$  and  $D'$ .
After  $x$  is removed, the response on  $D$  and  $D'$  is the same, and the data holder does not need to be called. Before  $x$  is removed from  $D'$ , we need to consider the queries such that  $f(x) \neq 0$  while  $C_x < \tau$ . Note that this is equivalent to a sequence of AboveThreshold tests to linear queries, we apply TCT analysis with ConditionalRelease applied with above threshold responses. The claim follows from Theorem B.4.
We also add that Algorithm 5 can be implemented with BetweenThresholds test (see Section 2.6), the extension is straightforward with the respective privacy bounds following from Lemma G.1 ( $q$  value for target hit).
J Private Selection
In this section we provide proofs and additional details for private selection in TCT (Sections 2.4 and 2.4.1). Let  $\mathcal{A}_1, \ldots, \mathcal{A}_m$  be of  $m$  private algorithms that return results with quality scores. The private selection task asks us to select the best algorithm from the  $m$  candidates. The one-shot selection described in Algorithm 3 (with  $k = 1$ ) runs each algorithm once and returns the response with highest quality.
It is shown in [21] that if each  $\mathcal{A}_i$  is  $(\varepsilon, 0)$ -DP then the one-shot selection algorithm degrades the privacy bound to  $(m\varepsilon, 0)$ -DP. However, if we relax the requirement to approximate DP, we can show that one-shot selection is  $(O(\log(1/\delta)\varepsilon), \delta)$ -DP, which is independent of  $m$  (the number of candidates). Moreover, in light of a lower-bound example by [21], Theorem J.1 is tight up to constant factors.
Formally, our theorem can be stated as
Theorem J.1. Suppose  $\varepsilon < 1$ . Let  $\mathcal{A}_1, \ldots, \mathcal{A}_m: X^n \to \mathcal{Y} \times \mathbb{R}$  be a list of  $(\varepsilon, \delta_i)$ -DP algorithms, where the output of  $\mathcal{A}_i$  consists of a solution  $y \in \mathcal{Y}$  and a score  $s \in \mathbb{R}$ . Denote by  $\operatorname{Best}(\mathcal{A}_1, \ldots, \mathcal{A}_m)$  the following algorithm (Algorithm 3 with  $k = 1$ ): run each  $\mathcal{A}_1, \ldots, \mathcal{A}_m$  once, get  $m$  results  $(y_1, s_1), \ldots, (y_m, s_m)$ , and output  $(y_{i^*}, s_{i^*})$  where  $i^* = \arg \max_i s_i$ .
Then, for every  $\delta \in (0,1)$ ,  $\operatorname{Best}(\mathcal{A}_1,\ldots ,\mathcal{A}_m)$  satisfies  $(\varepsilon^{\prime},\delta^{\prime})$  -DP where  $\varepsilon^{\prime} = O(\varepsilon \log (1 / \delta))$ ,  $\delta^{\prime} = \delta +\sum_{i}\delta_{i}$ .
Proof. Discrete scores. We start by considering the case that the output scores from  $\mathcal{A}_1, \ldots, \mathcal{A}_m$  always lie in a finite set  $X \subseteq \mathbb{R}$ . The case with continuous scores can be analyzed by a discretization argument.
Fix  $D^0, D^1$  to be a pair of adjacent data sets. We consider the following implementation of the vanilla private selection.
Algorithm 9: Private Selection: A Simulation
Input: Private data set  $D$  .The set  $X$  defined above.   
for  $i = 1,\dots ,m$  do  $\begin{array}{r}\lfloor (y_i,s_i)\leftarrow A_i(D) \end{array}$    
for  $\hat{s}\in X$  in the decreasing order do for  $i = 1,\ldots ,m$  do if  $s_i\geq \hat{s}$  then return  $(y_{i},s_{i})$
Assuming the score of  $\mathcal{A}_i(D)$  always lies in the set  $X$ , it is easy to see that Algorithm 9 simulates the top-1 one-shot selection algorithm (Algorithm 3 with  $k = 1$ ) perfectly. Namely, Algorithm 9 first runs each  $\mathcal{A}_i(D)$  once and collects  $m$  results. Then, the algorithm searches for the lowest  $\hat{s} \in X$  such that there is a pair  $(y_i, s_i)$  with a score of at least  $s_i \geq \hat{s}$ . The algorithm then publishes this score.
On the other hand, we note that Algorithm 9 can be implemented by the conditional release with revisions framework (Algorithm 2). Namely, Algorithm 9 first runs each private algorithm once and stores all the outcomes. Then the algorithm gradually extends the target set (namely, when the algorithm is searching for the threshold  $\hat{s}$ , the target set is  $\{(y,s):s\geq \hat{s}\}$ ), and tries to find an outcome in the target. Therefore, it follows from Lemma 2.5 and Theorem B.4 that Algorithm 9 is  $(O(\varepsilon \log (1 / \delta)),\delta +\sum_{i}\delta_{i})$  -DP.
Continuous scores. We then consider the case that the distributions of the scores of  $\mathcal{A}_1(D),\ldots ,\mathcal{A}_K(D)$  are continuous over  $\mathbb{R}$ . We additionally assume that the distribution has no "point mass". This is to say, for every  $i\in [m]$  and  $\hat{s}\in \mathbb{R}$ , it holds that
$$
\lim_{\Delta \to 0}\operatorname *{Pr}_{(y_{i},s_{i})\sim \mathcal{A}_{i}(D)}[\hat{s} -\Delta \leq s\leq \hat{s} +\Delta ] = 0.
$$
This assumption is without loss of generality because we can always add a tiny perturbation to the original output score of  $\mathcal{A}_i(D)$ .
Fix  $D, D'$  as two neighboring data sets. We show that the vanilla selection algorithm preserves differential privacy between  $D$  and  $D'$ .
Let  $\eta > 0$  be an arbitrarily small real number. Set  $M = \frac{10 \cdot m^4}{\eta}$ . For each  $\ell \in [1, M]$ , let  $q_\ell \in \mathbb{R}$  be the unique real such that
$$
\operatorname * {P r} _ {i \sim [ m ], (y _ {i}, s _ {i}) \sim \mathcal {A} _ {i} (D)} [ s _ {i} \geq q _ {\ell} ] = \frac {\ell}{M + 1}.
$$
Similarly we define  $q_{\ell}^{\prime}$  with respect to  $\mathcal{A}_i(D')$ . Let  $X = \{q_{\ell}, q_{\ell}'\}$ .
Now, consider running Algorithm 9 with the set  $X$  and candidate algorithms  $\mathcal{A}_1,\ldots ,\mathcal{A}_K$  on  $D$  or  $D^{\prime}$ . Sort elements of  $X$  in the increasing order, which we denote as  $X = \{\hat{q}_1\leq \dots \leq \hat{q}_m\}$ . After
sampling  $\mathcal{A}_i(D)$  for each  $i\in [m]$ , Algorithm 9 fails to return the best outcome only if one of the following events happens.
- The best outcome  $(y^{*}, s^{*})$  satisfies that  $s^{*} < \hat{q}_{1}$ .
- There are two outcomes  $(y_{i}, s_{i})$  and  $(y_{j}, s_{j})$  such that  $s_{i}, s_{j} \in [\hat{q}_{\ell}, \hat{q}_{\ell + 1}]$  for some  $\ell \in [n]$ .
If Item 1 happens, Algorithm 9 does not output anything. If Item 2 happens, then it might be possible that  $i < j$ ,  $s_i > s_j$ , but Algorithm 9 outputs  $s_i$ .
It is easy to see that Event 1 happens with probability at most  $\frac{m^2}{M} \leq \eta$  by the construction of  $X$ . Event 2 happens with probability at most  $M \cdot \frac{m^4}{M^2} \leq \eta$ . Therefore, the output distribution of Algorithm 9 differs from the true best outcome by at most  $O(\eta)$  in the statistical distance. Taking the limit  $\eta \to 0$  completes the proof.
Remark J.2. Theorem J.1 shows that there is a factor of  $\log(1/\delta)$  overhead when we run top-1 one-shot private selection (Algorithm 9) only once. Nevertheless, we observe that if we compose top-1 one-shot selection with other algorithms under the TCT framework (e.g., compose multiple top-1 one-shot selections, generalized private testing, or any other applications mentioned in this paper)), then on-average we only pay  $4\varepsilon$  privacy cost (one NotPrior target hit with a  $2\varepsilon$ -DP algorithm) per top-1 selection (assuming  $\varepsilon$  is sufficiently small so that  $e^{\varepsilon} \approx 1$ ). In particular, adaptively performing  $c$  executions of top-1 selection is  $(\varepsilon', \delta)$ -DP where  $\varepsilon' = \varepsilon \cdot (4\sqrt{c\log(1/\delta)} + o(\sqrt{c}))$ .
Liu and Talwar [21] established a lower bound of  $2\varepsilon$  on the privacy of a more relaxed top-1 selection task. Hence, there is a factor of 2 gap between this lower bound and our privacy analysis. Note that for the simpler task of one-shot above threshold score (discussed in Section 2.4.1), where the goal is to return a response that is above the threshold if there is one, can be implemented using a single target hit on Conditional Release call (without revise) and this matches the lower bound of  $2\varepsilon$ . We therefore suspect that it might be possible to tighten the privacy analysis of top-1 one-shot selection. We leave it as an interesting question for followup work.
J.1 One-Shot Top- $k$  Selection
In this section, we prove our results for top-  $k$  selection.
We consider the natural one-shot algorithm for top- $k$  selection described in Algorithm 3, which (as mentioned in the introduction) generalizes the results presented in [9, 29], which were tailored for selecting from 1-Lipschitz functions, using the Exponential Mechanism or the Report-Noise-Max paradigm.
We prove the following privacy theorem for Algorithm 3.
Theorem J.3. Suppose  $\varepsilon < 1$ . Assume that each  $\mathcal{A}_i$  is  $(\varepsilon, 0)$ -DP. Then, for every  $\delta \in (0, 1)$ , Algorithm 3 is  $(\varepsilon \cdot O(\sqrt{k \log(\frac{1}{\delta})} + \log(\frac{1}{\delta}))$ ,  $\delta)$ -DP.
Remark J.4. The constant hidden in the big-Oh depends on  $\varepsilon$ . For the setting that  $\varepsilon$  is close to zero so that  $e^{\varepsilon} \approx 1$  and  $\delta \geq 2^{o(k)}$ , the privacy bound is roughly  $(\varepsilon', \delta)$ -DP where  $\varepsilon' = \varepsilon \cdot (4\sqrt{k\log(1 / \delta)} + o(\sqrt{k}))$ .
Remark J.5. We can take  $\mathcal{A}_i$  as the Laplace mechanism applied to a 1-Lipschisz quality function  $f_i$  (namely,  $\mathcal{A}_i(D)$  outputs a pair  $(i, f_i(D) + \mathbf{Lap}(1 / \varepsilon))$ , where  $i$  denotes the ID of the  $i$ -th candidate, and  $f_i(D) + \mathbf{Lap}(1 / \varepsilon)$  is the noisy quality score of Candidate  $i$  with respect to the data  $D$ ). In this way, Theorem J.3 recovers the main result of [29].
Moreover, Theorem J.3 improves over [29] from three aspects: Firstly, Theorem J.3 allows us to report the noisy quality scores of selected candidates for free, while [29] needs to run one additional round of Laplace mechanism to publish the quality scores. Second, our privacy bound has no dependence on  $m$ , while the bound in the prior work [29] was  $(O(\varepsilon \sqrt{k\log(m / \delta)}), \delta)$ -DP. Lastly, Theorem J.3 applies more generally to any private-preserving algorithms, instead of the classic Laplace mechanism.
Proof. The proof is similar to that of Theorem J.1. Namely, we run each  $\mathcal{A}_i(D)$  once and store all results. Then we maintain a threshold  $T$ , which starts with  $T = \infty$ . We gradually decrease  $T$ ,
and use Algorithm 2 (Conditional Release with Revised Calls) to find outcomes with a quality score larger than  $T$ . We keep this process until we identify  $k$  largest outcomes. The claimed privacy bound now follows from Lemma 2.5 and Theorem B.4.