SlowGuess/NeurIPS_2023_OCR huggingface dataset:
David Durfee
Anonym Inc.
david@anonymco.com
In this work we consider the problem of differentially private computation of quantiles for the data, especially the highest quantiles such as maximum, but with an unbounded range for the dataset. We show that this can be done efficiently through a simple invocation of AboveThreshold, a subroutine that is iteratively called in the fundamental Sparse Vector Technique, even when there is no upper bound on the data. In particular, we show that this procedure can give more accurate and robust estimates on the highest quantiles with applications towards clipping that is essential for differentially private sum and mean estimation. In addition, we show how two invocations can handle the fully unbounded data setting. Within our study, we show that an improved analysis of AboveThreshold can improve the privacy guarantees for the widely used Sparse Vector Technique that is of independent interest. We give a more general characterization of privacy loss for AboveThreshold which we immediately apply to our method for improved privacy guarantees. Our algorithm only requires one  $O(n)$  pass through the data, which can be unsorted, and each subsequent query takes  $O(1)$  time. We empirically compare our unbounded algorithm with the state-of-the-art algorithms in the bounded setting. For inner quantiles, we find that our method often performs better on non-synthetic datasets. For the maximal quantiles, which we apply to differentially private sum computation, we find that our method performs significantly better.
In statistics, quantiles are values that divide the data into specific proportions, such as median that divides the data in half. Quantiles are a central statistical method for better understanding a dataset. However, releasing quantile values could leak information about specific individuals within a sensitive dataset. As a result, it becomes necessary to ensure that individual privacy is ensured within this computation. Differential privacy offers a rigorous method for measuring the amount that one individual can change the output of a computation. Due it's rigorous guarantees, differential privacy has become the gold standard for measuring privacy. This measurement method then offers an inherent tradeoff between accuracy and privacy with outputs of pure noise achieving perfect privacy. Thus, the goal of designing algorithms for differentially private quantile computation is to maximize accuracy for a given level of privacy.
There are a variety of previous methods for computing a given quantile of the dataset that we will cover in Section 1.2, but each of these requires known bounds on the dataset. The most effective and practical method invokes the exponential mechanism Smith (2011). For computing multiple quantiles this method can be called iteratively. Follow-up work showed that it could be called recursively by splitting the dataset at each call to reduce the privacy cost of composition Kaplan et al. (2022). Further, a generalization can be called efficiently in one shot Gillenwater et al. (2021).
In this work, we offer an alternative practical and accurate approach, Unbounded Quantile Estimation (UQE), that also invokes a well-known technique and can additionally be applied to the unbounded setting. While the commonly-used technique designs a distribution to draw from that is specific to the dataset, our method will simply perform a noisy guess-and-check. Initially we assume there is only a lower bound on the data, as non-negative data is common in real world datasets with sensitive individual information. Our method will simply iteratively increase the candidate value by a small percentage and halt when the number of data points below the value exceeds the desired amount dictated by the given quantile. While the relative increase will be small each iteration, the exponential nature still implies that the candidate value will become massive within a reasonable number of iterations. As a consequence, our algorithm can handle the unbounded setting where we also show that two calls to this procedure can handle fully unbounded data. Computing multiple quantiles can be achieved by applying the recursive splitting framework from Kaplan et al. (2022).
Performing our guess-and-check procedure with differential privacy exactly fits AboveThreshold, a method that is iteratively called in the Sparse Vector Technique Dwork et al. (2009). We also take a deeper look at AboveThreshold and unsurprisingly show that similar to report noisy max algorithms, the noise addition can come from the Laplace, Gumbel or Exponential distributions. We further push this analysis to show that for monotonic queries, a common class of queries which we will also utilize in our methods, the privacy bounds for composition within the Sparse Vector Technique can be further improved. Given the widespread usage of this technique, $^{1}$  we believe this result is of independent interest. Furthermore, we give a more general characterization of query properties that can improve the privacy bounds of AboveThreshold. We immediately utilize this characterization in our unbounded quantile estimation algorithm to improve privacy guarantees.
While the commonly used algorithms for quantile estimation can still apply incredibly loose bounds to ensure the data is contained within, this can have a substantial impact upon the accuracy for estimating the highest quantiles such as maximum. This leads to an especially important application for our algorithm, differentially private sum computation, which can thereby be used to compute mean as well. Performing this computation practically without assumptions upon the distribution often requires clipping the data and adding noise proportionally. Clipping too high adds too much noise, and clipping too low changes the sum of the data too much. The highest quantiles of the data are used for clipping to optimize this tradeoff. The unbounded nature of our approach fundamentally allows us to estimate the highest quantiles more robustly and improve the accuracy of differentially private sum computation.
This improvement in differentially private sum computation is further evidenced by our empirical evaluation, with significant improvements in accuracy. Our empirical comparison will be upon the same datasets from previous work in the bounded setting. We also compare private computation of the inner quantiles on these datasets. For synthetic datasets generated from uniform or gaussian distributions, we see that the more structured approach of designing a distribution for the data from the exponential mechanism consistently performs better. However, for the real-world datasets, we see that our unstructured approach tends to perform better even within this bounded setting. By design our algorithm is less specific to the data, so our alternative approach becomes advantageous when less is known about the structure and bounds of the data a priori. As such, for large-scale privacy systems that provide statistical analysis for a wide variety of datasets, our methods will be more flexible to handle greater generality accurately.
1.2 Background literature
The primary algorithm for privately computing a given quantile, by which we compare our technique, applies the exponential mechanism with a utility function based upon closeness to the true quantile Smith (2011). We will discuss this algorithm, which we denote as Exponential Mechanism Quantile (EMQ), in greater detail in Appendix A. This approach was then extended to computing multiple quantiles more cleverly by recursively splitting the data and establishing that only one partition of
<sup>1</sup>For example, see Roth and Roughgarden (2010); Hardt and Rothblum (2010); Dwork et al. (2015); Nissim et al. (2016); Nissim and Stemmer (2018); Kaplan et al. (2020a,b); Hasidim et al. (2020); Bun et al. (2017); Bassily et al. (2018); Cummings et al. (2020); Ligett et al. (2017); Barthe et al. (2016a,b); Steinke and Ullman (2016); Cummings et al. (2015); Ullman (2015); Nandi and Bassily (2020); Shokri and Shmatikov (2015); Hsu et al. (2013); Sajed and Sheffet (2019); Feldman and Steinke (2017); Blum et al. (2015); Chen et al. (2016)
the dataset can change between neighbors thereby reducing the composition costs Kaplan et al. (2022). Additional follow-up work showed that a generalization of the utility function to multiple quantiles could be efficiently drawn upon in one shot Gillenwater et al. (2021). Another recent result examined this problem in the streaming data setting and gave a method that only uses strongly sub-linear space complexity Alabi et al. (2022).
Quantile computation can also be achieved through CDF estimation Bun et al. (2015); Kaplan et al. (2020a). However these techniques offer limited practicality as they rely upon several reductions and parameter tuning. Recursively splitting the data is also done for CDF estimation algorithms where the statistics from each split can be aggregated for quantile computation Dwork et al. (2010); Chan et al. (2011). These techniques tend to be overkill for quantile estimation and thus suffer in accuracy comparatively.
We will also give improved privacy analysis of the Sparse Vector Technique which was originally introduced in Dwork et al. (2009). A more detailed analysis of the method can be found in Lyu et al. (2017). Additional recent work has shown that more information can be output from the method at no additional privacy cost Kaplan et al. (2021); Ding et al. (2023).
We provide the requisite notation and definitions in Section 2. In Section 3, we review the AboveThreshold algorithm from the literature and show that privacy analysis can be further improved. In Section 4, we provide our unbounded quantile estimation method. In Section 5, we test our method compared to the previous techniques on synthetic and real world datasets. In Appendix A, we consider the estimation of the highest quantiles which has immediate application to differentially private sum and mean estimation. In Appendix B, we give further results on the AboveThreshold algorithm and provide the missing proofs from Section 3. In Appendix C, we provide further variants and extensions of our unbounded quantile estimation technique.
We will let  $x, x'$  denote datasets in our data universe  $\mathcal{X}$ .
Definition 2.1. Datasets  $x, x' \in \mathcal{X}$  are neighboring if at most one individual's data has been changed.
Note that we use the swap definition, but our analysis of the AboveThresholdalgorithm will be agnostic to the definition of neighboring. Using this definition as opposed to the add-subtract definition is necessary to apply the same experimental setup as in Gillenwater et al. (2021). Our differentially private quantile estimation will apply to either and we will give the privacy guarantees if we instead use the add-subtract definition in Appendix C.2.
Definition 2.2. A function  $f: \mathcal{X} \to \mathbb{R}$  has sensitivity  $\Delta$  if for any neighboring datasets  $|f(x) - f(x')| \leq \Delta$
Definition 2.3. Dwork et al. (2006b,a) A mechanism  $M: \mathcal{X} \to \mathcal{Y}$  is  $(\varepsilon, \delta)$ -differentially-private (DP) if for any neighboring datasets  $x, x' \in \mathcal{X}$  and  $S \subseteq \mathcal{Y}$ :
$$
\Pr [ M (x) \in S ] \leqslant e ^ {\varepsilon} \Pr [ M \left(x ^ {\prime}\right) \in S ] + \delta .
$$
We will primarily work with pure differential privacy in this work where  $\delta = 0$ . We will also be considering the composition properties of the Sparse Vector Technique, and the primary method for comparison will be Concentrated Differential Privacy that has become widely used in practice due to it's tighter and simpler advanced composition properties Bun and Steinke (2016). This definition is instead based upon Reny divergence where for probability distributions  $P, Q$  over the same domain and  $\alpha > 1$
$$
D _ {\alpha} (P \| Q) = \frac {1}{\alpha - 1} \ln_ {z \sim P} \left[ \left(\frac {P (z)}{Q (z)}\right) ^ {\alpha - 1} \right]
$$
Definition 2.4. Bun and Steinke (2016) A mechanism  $M: \mathcal{X} \to \mathcal{Y}$  is  $\rho$ -zero-concentrated-differentially-private ( $zCDP$ ) if for any neighboring datasets  $x, x' \in \mathcal{X}$  and all  $\alpha \in (1,\infty)$ :
$$
D _ {\alpha} (M (x) \| M \left(x ^ {\prime}\right)) \leq \rho \alpha .
$$
We can translate DP into zCDP in the following way.
Proposition 1. Bun and Steinke (2016) If  $M$  satisfies  $\varepsilon$ -DP then  $M$  satisfies  $\frac{1}{2}\varepsilon^2 - zCDP$
In our examination of AboveThreshold we will add different types of noise, similar to the report noisy max algorithms Ding et al. (2021). Accordingly, we will consider noise from the Laplace, Gumbel and Exponential distributions where  $\mathsf{Lap}(b)$  has PDF  $p_{\mathrm{Lap}}(z; b)$ , Gumbel(b) has PDF  $p_{\mathrm{Gumbel}}(z; b)$ , and  $\mathsf{Expo}(b)$  has PDF  $p_{\mathrm{Expo}}(z; b)$  where
$$
p _ {\text {L a p}} (z; b) = \frac {1}{2 b} \exp (- | z | / b) \quad p _ {\text {G u m b e l}} (z; b) = \frac {1}{b} \exp \left(- \left(z / b + e ^ {- z / b}\right)\right)
$$
$$
p _ {\text {E x p o}} (z; b) = \left\{ \begin{array}{l l} \frac {1}{b} \exp (- z / b) & z \geq 0 \\ 0 & z <   0 \end{array} \right.
$$
We let  $\mathrm{Noise}(b)$  denote noise addition from any of  $\mathrm{Lap}(b)$ ,  $\mathrm{Gumbel}(b)$ , or  $\mathrm{Expo}(b)$ . We will also utilize the definition of the exponential mechanism to analyze the addition of Gumbel noise.
Definition 2.5. McSherry and Talwar (2007) The Exponential Mechanism is a randomized mapping  $M: \mathcal{X} \to \mathcal{Y}$  such that
$$
P r [ M (x) = y ] \propto \exp \left(\frac {\varepsilon \cdot q (x , y)}{2 \Delta}\right)
$$
where  $q:\mathcal{X}\times \mathcal{Y}\to \mathbb{R}$  has sensitivity  $\Delta$
3 Improved Analysis for Sparse Vector Technique
In this section, we review the AboveThreshold algorithm from the literature. To our knowledge, this technique has only been used with Laplace noise in the literature. Unsurprisingly, we show that Gumbel and Exponential noise can also be applied, with the former allowing for a closed form expression of each output probability. We further show that for monotonic queries the privacy analysis of the Sparse Vector Technique, which iteratively applies AboveThreshold, can be improved. All proofs are pushed to Appendix B where we also give a more general characterization of query properties that can improve the privacy bounds of AboveThreshold.
3.1 Above Threshold Algorithm
We first provide the algorithm for AboveThreshold where noise can be applied from any of the Laplace, Gumbel or Exponential distributions.
Algorithm 1 AboveThreshold
Require: Input dataset  $x$ , a stream of queries  $\{f_i : \mathcal{X} \to \mathbb{R}\}$  with sensitivity  $\Delta$ , and a threshold  $T$
1: Set  $\hat{T} = T + \mathrm{Noise}(\Delta /\varepsilon_1)$
2: for each query  $i$  do
3: Set  $\nu_{i} = \mathrm{Noise}(\Delta /\varepsilon_{2})$
4: if  $f_{i}(x) + \nu_{i} \geq \hat{T}$  then
5: Output T and halt
6: else
7: Output  $\bot$
8: end if
9: end for
We will also define a common class of queries within the literature that is often seen to achieve a factor of 2 improvement in privacy bounds.
Definition 3.1. We say that stream of queries  $\{f_i : \mathcal{X} \to \mathbb{R}\}$  with sensitivity  $\Delta$  is monotonic if for any neighboring  $x, x' \in \mathcal{X}$  we have either  $f_i(x) \leq f_i(x')$  for all  $i$  or  $f_i(x) \geq f_i(x')$  for all  $i$ .
To our knowledge all previous derivations of AboveThreshold in the literature apply Lap noise which gives the following privacy guarantees.
Lemma 3.1 (Theorem 2 and 3 of Lyu et al. (2017)). If the noise addition is  $\mathsf{Lap}$  then Algorithm 1 is  $(\varepsilon_{1} + 2\varepsilon_{2})$ -DP for general queries and is  $(\varepsilon_{1} + \varepsilon_{2})$ -DP for monotonic queries.
Given that Expo noise is one-sided Lap noise, it can often be applied for comparative algorithms such as this one and report noisy max as well. We will show this extension in the appendix for completeness.
Corollary 3.1. If the noise addition is Expo then Algorithm 1 is  $(\varepsilon_{1} + 2\varepsilon_{2})$  -DP for general queries and  $(\varepsilon_{1} + \varepsilon_{2})$  -DP for monotonic queries.
While the proofs for Expo noise generally follow from the Lap noise proofs, it will require different techniques to show that Gumbel noise can be applied as well. In particular, we utilize the known connection between adding Gumbel noise and the exponential mechanism.
Lemma 3.2. If the noise addition is Gumbel and  $\varepsilon_1 = \varepsilon_2$  then Algorithm 1 is  $(\varepsilon_1 + 2\varepsilon_2)$ -DP for general queries and  $(\varepsilon_1 + \varepsilon_2)$ -DP for monotonic queries.
We defer the proof of this to the appendix. In all of our empirical evaluations we will use Expo noise in our calls to AboveThreshold because it has the lowest variance for the same parameter. While we strongly believe that Expo noise will be most accurate under the same noise parameters, we leave a more rigorous examination to future work. We also note that this examination was implicitly done for report noisy max between Gumbel and Expo noise in McKenna and Sheldon (2020), where their algorithm is equivalent to adding Expo noise Ding et al. (2021), and Expo noise was shown to be clearly superior.
3.2 Improved privacy analysis for Sparse Vector Technique
In this section, we further consider the iterative application of AboveThreshold which is known as the sparse vector technique. We show that for monotonic queries, we can improve the privacy analysis of sparse vector technique to obtain better utility for the same level of privacy. Our primary metric for measuring privacy through composition will be zCDP which we defined in Section 2 and has become commonly used particularly due to the composition properties. We further show in the appendix that our analysis also enjoys improvement under the standard definition of differential privacy. These improved properties immediately apply to our unbounded quantile estimation algorithm as our queries will be monotonic.
Theorem 1. If the queries are monotonic, then for any noise addition of  $\text{Lap}$ , Gumbel, or Expo we have that Algorithm 1 is  $\frac{1}{2}\varepsilon^2 - z\text{CDP}$  where  $\varepsilon = \frac{\varepsilon_1}{2} + \varepsilon_2$ . If the noise addition is Gumbel then we further require  $\varepsilon_1 = \varepsilon_2$
Note that applying Proposition 1 will instead give  $\varepsilon = \varepsilon_{1} + \varepsilon_{2}$ . It will require further techniques to reduce this by  $\varepsilon_{1}/2$  which will immediately allow for better utility with the same privacy guarantees. This bound also follows the intuitive factor of 2 improvement that is often expected for monotonic queries. The analysis will be achieved through providing a range-bounded property, a definition that was introduced in Durfee and Rogers (2019). This definition is ideally suited to characterizing the privacy loss of selection algorithms, by which we can view AboveThreshold. As such it will also enjoy the improved composition bounds upon the standard differential privacy definition shown in Dong et al. (2020). This range bounded property was then unified with zCDP with improved privacy guarantees in Cesar and Rogers (2021).
We give a proof of this theorem along with further discussion in Appendix B.3. We will also give a generalized characterization of when we can take advantage of properties of the queries to tighten the privacy bounds in AboveThreshold. We will immediately utilize this characterization to improve the privacy guarantees for our method in Appendix C.
4 Unbounded Quantile Estimation
In this section we give our method for unbounded quantile estimation. We focus upon the lower bounded setting which we view as most applicable to real-world problems, where non-negative
data is incredibly common, particularly for datasets that contain information about individuals. This method can be symmetrically apply to upper bounded data, and in the appendix we will show how this approach can be extended to the fully unbounded setting.
For the quantile problem we will assume our data  $x \in \mathbb{R}^n$ . Given quantile  $q \in [0,1]$  and dataset  $x \in \mathbb{R}^n$  the goal is to find  $t \in \mathbb{R}$  such that  $|\{x_j \in x | x_j < t\}|$  is as close to  $qn$  as possible.
4.1 Unbounded quantile mechanism
The idea behind unbounded quantile estimation will be a simple guess-and-check method that will just invoke AboveThreshold. In particular, we will guess candidate values  $t$  such that  $|\{x_{j} \in x | x_{j} < t\}|$  is close to  $qn$ . We begin with the smallest candidate, recall that we assume lower bounded data here and generalize later, and iteratively increase by a small percentage. At each iteration we check if  $|\{x_{j} \in x | x_{j} < t\}|$  has exceeded  $qn$ , and terminate when it does, outputting the most recent candidate. In order to achieve this procedure privately, we will simply invoke AboveThreshold. We also discuss how this procedure can be achieved efficiently in Section 4.2.
Thus we give our algorithm here where the lower bound of the data,  $\ell$ , is our starting candidate and  $\beta$  is the scale at which the threshold increases. Given that we want our candidate value to increase by a small percentage it must start as a positive value. As such we will essentially just shift the data such that the lower bound is instead 1. In all our experiments we set  $\beta = 1.001$ , so the increase is by  $0.1\%$  each iteration.
Algorithm 2 Unbounded quantile mechanism
Require: Input dataset  $x$ , a quantile  $q$ , a lower bound  $\ell$ , and parameter  $\beta > 1$
1: Run AboveThreshold with  $x$ ,  $T = qn$  and  $f_{i}(x) = |\{x_{j}\in x|x_{j} - \ell +1 < \beta^{i}\}|$
2: Output  $\beta^k +\ell -1$  where  $k$  is the query that AboveThreshold halted at
Given that our method simply calls AboveThreshold it will enjoy all the privacy guarantees from Section 3.1. Furthermore we will show that our queries are monotonic.
Lemma 4.1. For any sequence of thresholds  $\{t_i\in \mathbb{R}\}$  let  $f_{i}(x) = |\{x_{j}\in x|x_{j} < t_{i}\} |$  for all  $i$ . For any neighboring dataset under Definition 2.1, we have that  $\{f_i\}$  are monotonic queries with sensitivity 1.
Proof. Let  $x_{j}$  be the value that differs between neighbors  $x, x'$ . Define  $S_{x,t} = \{x_{j} \in x | x_{j} < t\}$ . We consider the case  $x_{j}' > x_{j}$  and the other will follow symmetrically. For all thresholds  $t_{i} \in (-\infty, x_{j}]$  we have  $x_{j} \notin S_{x,t_{i}}$  and  $x_{j}' \notin S_{x',t_{i}}$ , so  $f_{i}(x) = f_{i}(x')$ . For all thresholds  $t_{i} \in (x_{j}',\infty)$  we have  $x_{j} \in S_{x,t_{i}}$  and  $x_{j}' \in S_{x',t_{i}}$ , so  $f_{i}(x) = f_{i}(x')$ . Finally, for all thresholds  $t_{i} \in (x_{j}, x_{j}')$  we have  $x_{j} \in S_{x,t_{i}}$  and  $x_{j}' \notin S_{x',t_{i}}$ , so  $f_{i}(x) = f_{i}(x') + 1$ . Therefore,  $f_{i}(x) \geq f_{i}(x')$  for all  $i$ , and the sensitivity is 1.
Note that for the swap definition of neighboring the threshold remains constant. We will discuss how to extend our algorithm and further improve the privacy bounds for the add-subtract definition of neighboring in the appendix.
4.2 Simple and scalable implementation
In this section we show how our call to AboveThreshold can be done with a simple linear time pass through the data, and each subsequent query takes  $O(1)$  time. While the running time could potentially be infinite, if we set  $\beta = 1.001$ , then after 50,000 iterations our threshold is already over  $10^{21}$  and thus highly likely to have halted. Unless the scale of the data is absurdly high or the  $\beta$  value chosen converges to 1, our guess-and-check process will finish reasonably quickly.