SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Algorithm 3: One-Shot Top-  $k$  Selection
Input: A dataset  $D$ . Candidate algorithms  $\mathcal{A}_1, \ldots, \mathcal{A}_m$ . Parameter  $k \leq m$ .  
 $S \gets \emptyset$   
for  $i = 1, \ldots, m$  do  
\[
\begin{array}{l} (y_i, s_i) \leftarrow \mathcal{A}_i(D) \\ S \leftarrow S \cup \{(i, y_i, s_i)\} \end{array}
\]  
return  $L \gets$  the top- $k$  triplets from  $S$ , by decreasing  $s_i$
The proofs of Lemma 2.6 and implications to selection tasks are provided in Section J. The proof utilizes Conditional Release with revisions (Section 2.3).
2.4.1 Selection using Conditional Release
We analyze private selection procedures using conditional release (see Section J for details). First note that ConditionalRelease calls (without revising) suffice for one-shot above-threshold selection (release all results with a quality score that exceeds a pre-specified threshold  $t$ ), with target hits only on what was released: We simply specify the release condition to be  $s_i > t$ . What is missing in order to implement one-shot top- $k$  selection is an ability to find the "right" threshold (a value  $t$  so that exactly  $k$  candidates have quality scores above  $t$ ), while incurring only  $k$  target hits. The revise calls provide the functionality of lowering the threshold of previous conditional release calls (lowering the threshold amounts to augmenting the target). This functionality allows us to simulate a sweep of the  $m$  results of the batch in the order of decreasing quality scores. We can stop the sweep when a certain condition is met (the condition must be based on the prefix of the ordered sequence that we viewed so far) and we incur target hits only for the prefix. To simulate a sweep, we run a high threshold conditional release of all  $m$  candidates and then incrementally lower the threshold using sets of  $m$  revise calls (one call per candidate). The released results are in decreasing order of quality scores. To prove Lemma 2.6 we observe that the one-shot top- $k$  selection (Algorithm 3) is simulated exactly by such a sweep that halts after  $k$  scores are released (the sweep is only used for analysis).
As mentioned, with this approach we can apply any stopping condition that depends on the prefix. This allows us to use data-dependent selection criteria. One natural such criteria (instead of using
a rigid value of  $k$ ) is to choose  $k$  when there is a large gap in the quality scores, that the  $(k + 1)$ st quality score is much lower than the  $k$ th score [35]. This criterion can be implemented using a one-shot algorithm and analyzed in the same way using an equivalent sweep. Data-dependent criteria are also commonly used in applications such as clustering (choose "the right" number of clusters according to gap in clustering cost) and greedy selection of representatives.
2.5 Best of multiple targets
Multi-target charging is a simple but useful extension of Algorithm 1 (that is "single target"). With  $k$ -TCT, queries have the form  $\left( \mathcal{A}, \left( \top_i \right)_{i \in [k]} \right)$  where  $\top_i$  for  $i \in [k]$  are  $q$ -targets (we allow targets to overlap). The algorithm maintains  $k$  counters  $(C_i)_{i \in [k]}$ . For each query, for each  $i$ , we increment  $C_i$  if  $r \in \top_i$ . We halt when  $\min_i C_i = \tau$ .
The multi-target extension allows us to flexibly reduce the total privacy cost to that of the "best" among  $k$  target indices in retrospect (the one that is hit the least number of times). Interestingly, this extension is almost free in terms of privacy cost: The number of targets  $k$  only multiplies the  $\delta$  privacy parameter (see Section B.1 for details).
2.6 BetweenThresholdsinTCT
The BetweenThresholds classifier refines the AboveThreshold test: The goal is to report if the noisy Lipschitz value is below, between, or above two thresholds  $t_{l} < t_{r}$ . We aim for privacy loss that only depends on between outcomes. An SVT-based BetweenThresholds was proposed by [3] (with noisy value and thresholds). Their analysis required the gap size to satisfy  $t_{r} - t_{l} \geq (12 / \varepsilon)(\log (10 / \varepsilon) + \log (1 / \delta) + 1)$ .
We consider the "natural" BetweenThresholds classifier that compares the Lipschitz value with added  $\mathbf{Lap}(1 / \varepsilon)$  noise to the two threshold values and reports the result. This is  $\varepsilon$ -DP and we show (see Section G) that the between outcome is a target with  $q \geq (1 - e^{-(t_r - t_l)\varepsilon}) \cdot \frac{1}{e^{\varepsilon} + 1}$ . This  $q$  value is at most that of a NotPrior target  $(\frac{1}{e^{\varepsilon} + 1})$  and degrades smoothly as the gap decreases. Importantly, there is almost no degradation for fairly small gaps: When  $t_r - t_l \geq 2 / \varepsilon$  (resp.  $3 / \epsilon$ ) the  $q$ -value is  $\approx \frac{0.87}{e^{\varepsilon} + 1}$  (resp  $\frac{0.95}{e^{\varepsilon} + 1}$ ).
One benefit of TCT BetweenThresholds is that it applies with much smaller gaps  $t_r - t_l$  compared with [3], also asymptotically. Another benefit that holds even for large gaps (where the SVT variant is applicable) is that the natural algorithm requires lower privacy noise for a given accuracy. In Section H we demonstrate this improvement numerically in that we can answer  $\times 6 - \times 95$  fold more queries for the same accuracy requirements compared to [3]. This brings BetweenThresholds into the practical regime.
We can compare an AboveThreshold test with a threshold  $t$  with a BetweenThresholds classifier with  $t_{l} = t - 1 / \varepsilon$  and  $t_{r} = t + 1 / \varepsilon$ . Surprisingly perhaps, despite BetweenThresholds being more informative than AboveThreshold, as it provides more granular information on the value, its privacy cost is lower, and is much lower when values are either well above or well below the thresholds. Somehow, the addition of the third between outcome to the test tightened the privacy analysis! A natural question is whether we can extend this benefit more generally – inject a “boundary outcome” as a target when our private algorithm does not have one, then tighten the privacy analysis when queries are “far” from the boundary. We introduce next a method that achieves this goal.
2.7 The Boundary Wrapper Method
When the algorithm is a tester or a classifier, the result is most meaningful when one outcome dominates the distribution  $\mathcal{A}(D)$ . Moreover, when performing a sequence of tests or classification tasks we might expect most queries to have high confidence labels (e.g., [27, 1]). Our hope then is to incur privacy cost that depends only on the "uncertainty," captured by the probability of nondominant outcomes. When we have for each computation a good prior on which outcome is most likely, this goal can be achieved via NotPrior targets (Section 2.1). When we expect the whole sequence to be dominated by one type of outcome, even when we don't know which one it is, this goal can be achieved via NotPrior with multiple targets (Section 2.5). But these approaches do not apply when a dominant outcome exists in most computations but we have no handle on it.
For a private test  $\mathcal{A}$ , can we choose a moving target per computation to be the value with the smaller probability  $\arg \min_{b\in \{0,1\}}\operatorname *{Pr}[\mathcal{A}(D) = b]$ ? More generally, with a private classifier, can we somehow choose the target to be all outcomes except for the most likely one? Our boundary wrapper, described in Algorithm 4, achieves that goal. The privacy wrapper  $\mathcal{W}$  takes any private algorithm  $\mathcal{A}$  such as a tester or a classifier, and wraps it to obtain algorithm  $\mathcal{W}(\mathcal{A})$ . The wrapped algorithm has its outcome set augmented to include one boundary outcome  $\top$  that is designed to be a  $q$ -target. The wrapper returns  $\top$  with some probability that depends on the distribution of  $\mathcal{A}(D)$  and otherwise returns a sample from  $\mathcal{A}(D)$  (that is, the output we would get when directly applying  $\mathcal{A}$  to  $D$ ). We then analyse the wrapped algorithm in TCT.
Note that the probability of the wrapper  $\mathcal{A}$  returning  $\top$  is at most  $1/3$  and is roughly proportional to the probability of sampling an outcome other than the most likely from  $\mathcal{A}(D)$ . When there is no dominant outcome the  $\top$  probability tops at  $1/3$ . Also note that a dominant outcome (has probability  $p \in [1/2, 1]$  in  $\mathcal{A}(D)$ ) has probability  $p/(2 - p)$  to be reported. This is at least  $1/3$  when  $p = 1/2$  and is close to 1 when  $p$  is close to 1. For the special case of  $\mathcal{A}$  being a private test, there is always a dominant outcome.
A wrapped AboveThreshold test provides the benefit of BetweenThresholds discussed in Section 2.6 where we do not pay privacy cost for values that are far from the threshold (on either side). This is achieved mechanically without the need to explicitly introduce two thresholds around the given one and defining a different algorithm.
Algorithm 4: Boundary Wrapper   
Input: Dataset  $D = \{x_{1},\ldots ,x_{n}\} \in X^{n}$  , a private algorithm A  $r^* \gets \arg \max_r\operatorname *{Pr}[\mathcal{A}(D) = r]$  // The most likely outcome of  $\mathcal{A}(D)$ $\pi (D)\gets 1 - \mathrm{Pr}[\mathcal{A}(D) = r^{*}]$  // Probability that  $\mathcal{A}$  does not return the most likely outcome   
 $c\sim \mathbf{Ber}(\min \left\{\frac{1}{3},\frac{\pi}{1 + \pi}\right\})$  // Coin toss for boundary   
if  $c = 1$  then Return T else Return  $\mathcal{A}(D)$  // return boundary or value
We show (proofs provided in Section E) that the wrapped algorithm is nearly as private as its baseline:
Lemma 2.7 (Privacy of a wrapped algorithm). If  $\mathcal{A}$  is  $\varepsilon$ -DP then Algorithm 4 applied to  $\mathcal{A}$  is  $t(\varepsilon)$ -DP where  $t(\varepsilon) \leq \frac{4}{3}\varepsilon$ .
Lemma 2.8 (q-value of the boundary target). The outcome  $\top$  of a boundary wrapper (Algorithm 4) of an  $\varepsilon$ -DP algorithm is a  $\frac{e^{t(\varepsilon)} - 1}{2(e^{\varepsilon + t(\varepsilon)} - 1)}$ -target.
For small  $\varepsilon$  we obtain  $q\approx t(\varepsilon) / (2(\varepsilon +t(\varepsilon))$ . Substituting  $t(\varepsilon) = \frac{4}{3}\varepsilon$  we obtain  $q\approx \frac{2}{7}$ . Since the target  $\top$  has probability at most  $1 / 3$ , this is a small loss of efficiency (1/6 factor overhead) compared with composition in the worst case when there are no dominant outcomes.
The boundary wrapper yields light-weight privacy analysis that pays only for the "uncertainty" of the response distribution  $\mathcal{A}(D)$  and can be an alternative to more complex approaches based on smooth sensitivity (the stability of  $\mathcal{A}(D)$  to changes in  $D$ ) [25, 10, 33]. Note that the boundary-wrapper method assumes availability of the probability of the most dominant outcome in the distribution  $\mathcal{A}(D)$ , when it is large enough. The probability can always be computed without incurring privacy costs (only computation cost) and is readily available with the Exponential Mechanism [24] or when applying known noise distributions for AboveThreshold, BetweenThresholds, and Report-Noise-Max [9]. In Section F we propose a boundary-wrapper that only uses sampling access to  $\mathcal{A}(D)$ .
2.7.1 Applications to Private Learning using Non-privacy-preserving Models
Methods that achieve private learning through training non-private models include Private Aggregation of Teacher Ensembles (PATE) [26, 27] and Model-Agnostic private learning [1]. The private dataset  $D$  is partitioned into  $k$  parts  $D = D_{1} \sqcup \dots \sqcup D_{k}$  and a model is trained (non-privately) on each part. For multi-class classification with  $c$  labels, the trained models can be viewed as functions  $\{f_{i} : \mathcal{X} \to [c]\}_{i \in [k]}$ . Note that changing one sample in  $D$  can only change the training set of one of the models. To privately label an example  $x$  drawn from a public distribution, we compute the
predictions of all the models  $\{f_i(x)\}_{i\in [k]}$  and consider the counts  $n_j = \sum_{i\in [k]}\mathbf{1}\{f_i(x) = j\}$  (the number of models that gave label  $j$  to example  $x$ ) for  $j\in [c]$ . We then privately aggregate to obtain a private label, for example using the Exponential Mechanism [24] or Report-Noisy-Max [9, 29]. This setup is used to process queries (label examples) until the privacy budget is exceeded. In PATE, the new privately-labeled examples are used to train a new student model (and  $\{f_i\}$  are called teacher models). In these applications we seek tight privacy analysis. Composition over all queries - for  $O(1)$  privacy, only allows for  $O(k^2)$  queries. We aim to replace this with  $O(k^2)$  "target hits." These works used a combination of methods including SVT, smooth sensitivity, distance-to-instability, and propose-test-release [10, 33]. The TCT toolkit can streamline the analysis:
(i) It was noted in [1, 27] that when the teacher models are sufficiently accurate, we can expect that  $n_j \gg k/2$  on the ground truth label  $j$  on most queries. High-agreement examples are also more useful for training the student model. Moreover, agreement implies stability and lower privacy cost (when accounted through the mentioned methods) is lower. Instead, to gain from this stability, we can apply the boundary wrapper (Algorithm 4) on top of the Exponential Mechanism. Then use  $\top$  as our target. Agreement queries, where  $\max_j n_j \gg k/2$  (or more finely, when  $h = \arg \max_j n_j$  and  $n_h \gg \max_{j \in [k] \setminus \{h\}} n_j$ ) are very unlikely to be target hits.
(ii) If we expect most queries to be either high agreement  $\max_j n_j \gg k/2$  or no agreement  $\max_j n_j \ll k/2$  and would like to avoid privacy charges also with no agreement, we can apply AboveThreshold test to  $\max_j n_j$ . If above, we apply the exponential mechanism. Otherwise, we report "Low." The wrapper applied to the combined algorithm returns a label in  $[c]$ , "Low," or  $\top$ . Note that "Low" is a dominant outcome with no-agreement queries (where the actual label is not useful anyway) and a class label in  $[c]$  is a dominant outcome with high agreement. We therefore incur privacy loss only on weak agreements.
2.8 SVT with individual privacy charging
Our TCT privacy analysis simplifies and improves the analysis of SVT with individual privacy charging, introduced by Kaplan et al [20]. The input is a dataset  $D \in \mathcal{X}^n$  and an online sequence of linear queries that are specified by predicate and threshold value pairs  $(f_i, T_i)$ . For each query, the algorithms report noisy AboveThreshold test results  $\sum_{x \in D} f_i(x) \gtrsim T$ . Compared with the standard SVT, which halts after reporting  $\tau$  positive responses, SVT with individual charging maintains a separate budget counter  $C_x$  for each item  $x$ . For each query with a positive response, the algorithm only charges items that contribute to this query (namely, all the  $x$ 's such that  $f_i(x) = 1$ ). Once an item  $x$  contributes to  $\tau$  hits (that is,  $C_x = \tau$ ), it is removed from the data set. This finer privacy charging improves utility with the same privacy budget, as demonstrated by several recent works [20, 6]. We establish the following:
Theorem 2.9 (Privacy of Algorithm 5). Assume  $\varepsilon < 1$ . Algorithm 5 is  $(O(\sqrt{\tau \log(1 / \delta)} \varepsilon, 2^{-\Omega(\tau)} + \delta)$ -DP for every  $\delta \in (0,1)$ .
Compared with prior work [20]: Algorithm 5 simply adds Laplace noise to obtain  $\hat{f}_i = \left(\sum_{x\in D}f_i(x)\right) + \mathbf{L}\mathbf{a}\mathbf{p}(1 / \varepsilon)$  and then tests whether  $\hat{f}_i\geq T$  , whereas [20] adds two independent Laplace noises and publishes the approximate sum  $\hat{f}_i$  for "Above-Threshold" without incurring additional privacy loss. Our analysis is much simpler (few lines instead of several pages) and significantly tighter, also asymptotically: For the privacy bound in the statement of Theorem 5, our additive error is lower by a  $\log (1 / \varepsilon)\sqrt{\log(1 / \delta)}$  factor. Importantly, our improvement aligns the bounds of SVT with individual privacy charging with those of standard SVT, bringing the former into the practical regime. See Section I for details.
Conclusion We introduced the Target Charging Technique (TCT), a versatile unified privacy analysis framework that is particularly suitable when a sensitive dataset is accessed multiple times via differentially private algorithms. We provide an expansive toolkit and demonstrate significant improvement over prior work for basic tasks such as private testing and one-shot selection, describe use cases, and list challenges for followup works. TCT is simple with low overhead and we hope will be adopted in practice.
Algorithm 5: SVT with Individual Privacy Charging
Input: Private data set  $D\in \mathcal{X}^n$  ; privacy budget  $\tau >0$  ; Privacy parameter  $\varepsilon >0$    
foreach  $x\in D$  do  $C_x\gets 0$  // Initialize a counter for item  $x$    
for  $i = 1,2,\ldots ,$  do //Receive queries   
Receive a predicate  $f_{i}:\mathcal{X}\to [0,1]$  and threshold  $T_{i}\in \mathbb{R}$ $\hat{f}_i\gets (\sum_{x\in D}f_i(x)) + \mathrm{Lap}(1 / \varepsilon)$  // Add Laplace noise to count   
if  $\hat{f}_i\geq T_i$  then // Compare with threshold   
Publish  $\hat{f}_i$    
foreach  $x\in D$  such that  $f(x) > 0$  do   
 $C_x\gets C_x + 1$  if  $C_x = \tau$  then Remove  $x$  from  $D$    
else Publish  $\perp$
Acknowledgement Edith Cohen is partially supported by Israel Science Foundation (grant no. 1156/23)
[1] Raef Bassily, Om Thakkar, and Abhradeep Guha Thakurta. Model-agnostic private learning. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018.
[2] Amos Beimel, Haim Kaplan, Yishay Mansour, Kobbi Nissim, Thatchaphol Saranurak, and Uri Stemmer. Dynamic algorithms against an adaptive adversary: Generic constructions and lower bounds. CoRR, abs/2111.03980, 2021.
[3] Mark Bun, Thomas Steinke, and Jonathan Ullman. Make Up Your Mind: The Price of Online Queries in Differential Privacy, pages 1306-1325. 2017.
[4] H. Chernoff. A measure of the asymptotic efficiency for test of a hypothesis based on the sum of observations. Annals of Math. Statistics, 23:493-509, 1952.
[5] E. Cohen. Hyperloglog hyper extended: Sketches for concave sublinear frequency statistics. In KDD. ACM, 2017. full version: https://arxiv.org/abs/1607.06517.
[6] Edith Cohen, Xin Lyu, Jelani Nelson, Tamás Sarlós, Moshe Shechner, and Uri Stemmer. On the robustness of countsketch to adaptive inputs. In Proceedings of the 39th International Conference on Machine Learning (ICML), 2022.
[7] Edith Cohen, Xin Lyu, Jelani Nelson, Tamás Sarlós, and Uri Stemmer. Generalized Private Selection and Testing with High Confidence. In 14th Innovations in Theoretical Computer Science Conference (ITCS 2023), volume 251 of Leibniz International Proceedings in Informatics (LIPics), Dagstuhl, Germany, 2023. Schloss Dagstuhl – Leibniz-Zentrum für Informatik.
[8] Edith Cohen, Xin Lyu, Jelani Nelson, Tamás Sarlós, and Uri Stemmer. Optimal differentially private learning of thresholds and quasi-concave optimization. In Proceedings of the 55th Annual ACM Symposium on Theory of Computing, STOC 2023, page 472-482, New York, NY, USA, 2023. Association for Computing Machinery.
[9] David Durfee and Ryan M. Rogers. Practical differentially private top-k selection with pay-what-you-get composition. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d'Alché-Buc, Emily B. Fox, and Roman Garnett, editors, Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pages 3527-3537, 2019.
[10] Cynthia Dwork and Jing Lei. Differential privacy and robust statistics. STOC '09, New York, NY, USA, 2009. Association for Computing Machinery.
[27] Nicolas Papernot, Shuang Song, Ilya Mironov, Ananth Raghunathan, Kunal Talwar, and Ülfar Erlingsson. Scalable private learning with PATE. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenReview.net, 2018.
[28] Nicolas Papernot and Thomas Steinke. Hyperparameter tuning with Rényi differential privacy. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022.
[29] Gang Qiao, Weijie J. Su, and Li Zhang. Oneshot differentially private top-k selection. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pages 8672-8681. PMLR, 2021.
[30] Aaron Roth and Tim Roughgarden. Interactive privacy via the median mechanism. In Leonard J. Schulman, editor, Proceedings of the 42nd ACM Symposium on Theory of Computing, STOC 2010, Cambridge, Massachusetts, USA, 5-8 June 2010, pages 765-774. ACM, 2010.
[31] Adam Smith, Shuang Song, and Abhradeep Thakurta. The flajolet-martin sketch itself preserves differential privacy: Private counting with minimal space. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS'20, 2020.
[32] Thomas Steinke and Jonathan R. Ullman. Tight lower bounds for differentially private selection. In Chris Umans, editor, 58th IEEE Annual Symposium on Foundations of Computer Science, FOCS 2017, Berkeley, CA, USA, October 15-17, 2017, pages 552-563. IEEE Computer Society, 2017.
[33] Abhradeep Guha Thakurta and Adam Smith. Differentially private feature selection via stability arguments, and the robustness of the lasso. In Shai Shalev-Shwartz and Ingo Steinwart, editors, Proceedings of the 26th Annual Conference on Learning Theory, volume 30 of Proceedings of Machine Learning Research, pages 819-850, Princeton, NJ, USA, 12-14 Jun 2013. PMLR.
[34] Salil P. Vadhan. The complexity of differential privacy. In Yehuda Lindell, editor, Tutorials on the Foundations of Cryptography, pages 347-450. Springer International Publishing, 2017.
[35] Yuqing Zhu and Yu-Xiang Wang. Adaptive private-k-selection with adaptive k and application to multi-label pate. In Gustau Camps-Valls, Francisco J. R. Ruiz, and Isabel Valera, editors, Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, volume 151 of Proceedings of Machine Learning Research, pages 5622-5635. PMLR, 28-30 Mar 2022.
Notation. We say that a function  $f$  over datasets is  $t$ -Lipschitz if for any two neighboring datasets  $D^0, D^1$ , it holds that  $|f(D^1) - f(D^0)| \leq t$ . For two reals  $a, b \geq 0$  and  $\varepsilon > 0$ , we write  $a \approx_{\varepsilon} b$  if  $e^{-\varepsilon}b \leq a \leq e^{\varepsilon}b$ .
For two random variables  $X^0, X^1$ , we say that they are  $\varepsilon$ -indistinguishable, denoted  $X^0 \approx_{\varepsilon} X^1$ , if their max-divergence and symmetric counterpart are both at most  $\varepsilon$ . That is, for  $b \in \{0, 1\}$ ,  $\max_{S \subseteq \operatorname{supp}(X^b)} \ln \left[\frac{\operatorname*{Pr}[X^b \in S]}{\operatorname*{Pr}[X^{1 - b} \in S]}\right] \leq \varepsilon$ .
We similarly say that for  $\delta > 0$ , the random variables are  $(\varepsilon, \delta)$ -indistinguishable, denoted  $X^0 \approx_{\varepsilon, \delta} X^1$ , if for  $b \in \{0, 1\}$
$$
\max  _ {S \subseteq \operatorname {s u p p} (X ^ {b})} \ln \left[ \frac {\Pr [ X ^ {b} \in S ] - \delta}{\Pr [ X ^ {1 - b} \in S ]} \right] \leq \varepsilon .
$$
For two probability distributions,  $\mathcal{B}^0$ ,  $\mathcal{B}^1$  We extend the same notation and write  $\mathbf{B}^0 \approx_{\varepsilon} \mathbf{B}^1$  and  $\mathbf{B}^0 \approx_{\varepsilon, \delta} \mathbf{B}^1$  when this holds for random variables drawn from the respective distributions.
The following relates  $(\varepsilon, 0)$  and  $(\varepsilon, \delta)$ -indistinguishability with  $\delta = 0$  and  $\delta > 0$ .