SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\varepsilon \left(x, x ^ {\prime}\right) = \max  _ {k} \left(\frac {\varepsilon_ {1}}{\Delta} \Delta_ {k} \left(x, x ^ {\prime}\right) + \frac {\varepsilon_ {2}}{\Delta} \max  \{0, \Delta_ {k} \left(x, x ^ {\prime}\right) - \left(f _ {k} \left(x ^ {\prime}\right) - f _ {k} (x)\right) \}\right)
$$
where  $\Delta_k(x,x') = \max_{i < k} \max \{0, f_i(x') - f_i(x)\}$ .
Given that our definition is meant to encompass the one-sided privacy loss for AboveThreshold we will now prove that fact for Expo and Gumbel noise (and Lap follows equivalently to Expo). We first prove this conjecture for Expo noise.
Lemma B.1. For a stream of queries  $\{f_i\}$  with sensitivity  $\Delta$ , threshold  $T$  and neighboring data sets  $x, x'$ , if we run Algorithm 1 with Expo noise, then for any given outcome  $\{\bot^{k-1}, \top\}$  we have
$$
\begin{array}{l} \Pr \left[ \text {A b o v e T h r e s h o l d} (x, \{f _ {i} \}, T) = \{\bot^ {k - 1}, \top \} \right] \leq \\ \exp (\varepsilon (x, x ^ {\prime})) \Pr \left[ \text {A b o v e T h r e s h o l d} \left(x ^ {\prime}, \left\{f _ {i} ^ {\prime} \right\}, T\right) = \{\bot^ {k - 1}, \top \} \right] \\ \end{array}
$$
Proof. Let  $\nu_{i} \sim \mathrm{Exp o}(\Delta/\varepsilon_{2})$  denote the noise drawn for query  $f_{i}$ , and let  $\nu \sim \mathrm{Exp o}(\Delta/\varepsilon_{1})$  denote the noise drawn for the threshold. Further let  $\nu_{i \leq k}$  denote all  $\nu_{i}$  such that  $i \leq k$ . By construction, we have
$$
\Pr \left[ \text {A b o v e T h r e s h o l d} (x, \{f _ {i} \}, T) = \{\bot^ {k - 1}, \top \} \right] = \Pr_ {\nu , \nu_ {i \leq k}} \left[ \max  _ {i <   k} f _ {i} (x) + \nu_ {i} <   T + \nu <   f _ {k} (x) + \nu_ {k} \right]
$$
We will fix the randomness of  $\nu_{i < k}$ , denoting  $\nu_{i}$  such that  $i < k$ , for datasets  $x$  and  $x'$ , and let  $\tau = \max_{i < k} f_{i}(x) + \nu_{i}$  and  $\tau' = \max_{i < k} f_{i}(x') + \nu_{i}$ . It suffices then to show that for any fixed randomness  $\nu_{i < k}$  we have
$$
\Pr_ {v, v _ {k}} \left[ \tau <   T + v <   f _ {k} (x) + v _ {k} \right] \leq \exp \left(\varepsilon \left(x, x ^ {\prime}\right)\right) \Pr_ {v, v _ {k}} \left[ \tau^ {\prime} <   T + v <   f _ {k} \left(x ^ {\prime}\right) + v _ {k} \right]
$$
Let  $\Delta_1 = \max \{0, \tau' - \tau\}$  and  $\Delta_2 = \max \{0, \Delta_1 - (f_k(x') - f_k(x))\}$  and let  $\nu' = \nu + \Delta_1$  and  $\nu'_k = \nu_k + \Delta_2$ . It suffices to show that for every pair of draws  $\nu, \nu_k$  that satisfies  $\tau < T + \nu < f_k(x) + \nu_k$ , then the corresponding  $\nu', \nu'_k$  are i) non-negative, ii) satisfy  $\tau' < T + \nu' < f_k(x') + \nu'_k$  and iii)
$$
p _ {\text {E x p o}} \left(v; \Delta / \varepsilon_ {1}\right) \cdot p _ {\text {E x p o}} \left(v _ {k}; \Delta / \varepsilon_ {2}\right) \leq \exp \left(\varepsilon \left(x, x ^ {\prime}\right)\right) p _ {\text {E x p o}} \left(v ^ {\prime}; \Delta / \varepsilon_ {1}\right) \cdot p _ {\text {E x p o}} \left(v _ {k} ^ {\prime}; \Delta / \varepsilon_ {2}\right)
$$
We note that condition iii) does not require any scalar on the RHS as our change of variable is a shift for each so the Jacobian of the mapping is the identity matrix with determinant of 1.
Condition i) follows from the fact that  $\nu, \nu_{k}$  are non-negative because they're drawn from the exponential distribution, and  $\Delta_1, \Delta_2$  are non-negative by construction.
For condition ii), if  $\tau < T + \nu$  we must have  $\tau' < T + \nu + \Delta_1$  by definition of  $\Delta_1$ . Similarly, if  $T + \nu < f_k(x) + \nu_k$  then  $T + \nu + \Delta_1 < f_k(x') + \nu_k + \Delta_2$  because  $\Delta_2 \geq \Delta_1 - (f_k(x') - f_k(x))$ .
For condition iii), the PDF of the exponential distribution implies  $p_{\mathrm{Expo}}(\nu; \Delta / \varepsilon_1) \leq \exp \left( \frac{\varepsilon_1 \Delta_1}{\Delta} \right) p_{\mathrm{Expo}}(\nu'; \Delta / \varepsilon_1)$  and  $p_{\mathrm{Expo}}(\nu_k; \Delta / \varepsilon_2) \leq \exp \left( \frac{\varepsilon_2 \Delta_2}{\Delta} \right) p_{\mathrm{Expo}}(\nu'_k; \Delta / \varepsilon_2)$ . Due to the fixing of randomness of  $\nu_{i < k}$ , we have that  $\tau' - \tau \leq \max_{i < k} (f_i(x') - f_i(x))$  so  $\Delta_1 \leq \Delta_k(x, x')$  from Definition B.1 and  $\Delta_2 \leq \max \{0, \Delta_k(x, x') - (f_k(x') - f_k(x))\}$  which implies condition iii).
This proof then easily applies to Lap as well. The proof for Gumbel will differ though as we no longer have similar closeness of PDF properties, but it will follow from the fact that we can give a closed form expression for each probability.
Lemma B.2. Given a dataset  $x$ , a stream of queries  $\{f_i\}$  and threshold  $T$ , for any outcome  $\{\bot^{k-1}, \top\}$  from running AboveThreshold with Gumbel noise and  $\varepsilon = \varepsilon_1 = \varepsilon_2$ , then
$$
\begin{array}{l} P r \left[ \text {A b o v e T h r e s h o l d} (x, \{f _ {i} \}, T) = \{\bot^ {k - 1}, \top \} \right] = \\ \frac {\exp \left(\frac {\varepsilon}{\Delta} f _ {k} (x)\right)}{\exp \left(\frac {\varepsilon}{\Delta} T\right) + \sum_ {i = 1} ^ {k} \exp \left(\frac {\varepsilon}{\Delta} f _ {i} (x)\right)} \cdot \frac {\exp \left(\frac {\varepsilon}{\Delta} T\right)}{\exp \left(\frac {\varepsilon}{\Delta} T\right) + \sum_ {i = 1} ^ {k - 1} \exp \left(\frac {\varepsilon}{\Delta} f _ {i} (x)\right)} \\ \end{array}
$$
Proof. We add noise  $Gumbel(\Delta/\varepsilon)$  to all  $f_i(x)$  and the threshold  $T$ , and in order to output  $\{\bot^{k-1}, \top\}$  we must have that for the first  $k$  queries, the noisy value of  $f_k(x)$  is the largest and the noisy threshold is the second largest. It is folklore in the literature that adding Gumbel noise (with the same noise parameter) and choosing the largest index is equivalent to the exponential mechanism. This can also be extended to showing that adding Gumbel noise and choosing the top-k indices in order is equivalent to the peeling exponential mechanism. The peeling exponential mechanism first selects the top index using the exponential mechanism and removes it from the candidate set, repeating iteratively until accumulating the top-k indices. A formal proof of this folklore result is also provided in Lemma 4.2 of Durfee and Rogers (2019). Applying this result and the definition of the exponential mechanism gives the desired equality.
We then utilize this closed form expression to get our desired one-sided privacy bounds when using Gumbel noise.
Lemma B.3. For a stream of queries  $\{f_i\}$  with sensitivity  $\Delta$ , threshold  $T$  and neighboring data sets  $x, x'$ , if we run Algorithm 1 with Gumbel noise with  $\varepsilon_1 = \varepsilon_2$ , then for any outcome  $\{\bot^{k-1}, \top\}$  we have
$$
\begin{array}{l} P r \left[ \text {A b o v e T h r e s h o l d} (x, \{f _ {i} \}, T) = \{\bot^ {k - 1}, \top \} \right] \leq \\ \exp (\varepsilon (x, x ^ {\prime})) P r \left[ \text {A b o v e T h r e s h o l d} \left(x ^ {\prime}, \left\{f _ {i j} \right\}, T\right) = \{\bot^ {k - 1}, \top \} \right] \\ \end{array}
$$
Proof. Let  $\varepsilon = \varepsilon_{1} = \varepsilon_{2}$ . From Lemma B.2 we know that
$$
\begin{array}{l} \Pr \left[ \text {A b o v e T h r e s h o l d} (x, \{f _ {i} \}, T) = \{\bot^ {k - 1}, \top \} \right] = \\ \frac {\exp \left(\frac {\varepsilon}{\Delta} f _ {k} (x)\right)}{\exp \left(\frac {\varepsilon}{\Delta} T\right) + \sum_ {i = 1} ^ {k} \exp \left(\frac {\varepsilon}{\Delta} f _ {i} (x)\right)} \cdot \frac {\exp \left(\frac {\varepsilon}{\Delta} T\right)}{\exp \left(\frac {\varepsilon}{\Delta} T\right) + \sum_ {i = 1} ^ {k - 1} \exp \left(\frac {\varepsilon}{\Delta} f _ {i} (x)\right)} \\ \end{array}
$$
Similar to the proof of Lemma B.1, we let  $\Delta_1 = \max \{0, \max_{i < k} (f_i(x') - f_i(x))\}$  and  $\Delta_2 = \max \{0, \Delta_1 + (f_k(x) - f_k(x'))\}$ . We first want to show that
$$
\frac {\exp \left(\frac {\varepsilon}{\Delta} T\right)}{\exp \left(\frac {\varepsilon}{\Delta} T\right) + \sum_ {i = 1} ^ {k - 1} \exp \left(\frac {\varepsilon}{\Delta} f _ {i} (x)\right)} \leq e ^ {\frac {\varepsilon \Delta_ {1}}{\Delta}} \frac {\exp \left(\frac {\varepsilon}{\Delta} T\right)}{\exp \left(\frac {\varepsilon}{\Delta} T\right) + \sum_ {i = 1} ^ {k - 1} \exp \left(\frac {\varepsilon}{\Delta} f _ {i} \left(x ^ {\prime}\right)\right)} \tag {1}
$$
This inequality reduces to showing  $\exp (\frac{\epsilon}{\Delta} T) + \sum_{i = 1}^{k - 1}\exp (\frac{\epsilon}{\Delta} f_i(x'))\leq e^{\frac{\epsilon\Delta_1}{\Delta}}(\exp (\frac{\epsilon}{\Delta} T) + \sum_{i = 1}^{k - 1}\exp (\frac{\epsilon}{\Delta} f_i(x)))$  This holds from the fact that  $\exp (\frac{\epsilon}{\Delta} T)\leq \exp (\epsilon \Delta_1 / \Delta)\exp (\frac{\epsilon}{\Delta} T)$  because  $\epsilon \Delta_1 / \Delta \geq 0$  and  $\exp (\frac{\epsilon}{\Delta} f_i(x'))\leq$ $\exp (\epsilon \Delta_1 / \Delta)\exp (\frac{\epsilon}{\Delta} f_i(x))$  for all  $i <   k$  by our definition of  $\Delta_{1}$
$$
\frac {\exp \left(\frac {\varepsilon}{\Delta} f _ {k} (x)\right)}{\exp \left(\frac {\varepsilon}{\Delta} T\right) + \sum_ {i = 1} ^ {k} \exp \left(\frac {\varepsilon}{\Delta} f _ {i} (x)\right)} \leq e ^ {\frac {\varepsilon \Delta_ {2}}{\Delta}} \frac {\exp \left(\frac {\varepsilon}{\Delta} f _ {k} \left(x ^ {\prime}\right)\right)}{\exp \left(\frac {\varepsilon}{\Delta} T\right) + \sum_ {i = 1} ^ {k} \exp \left(\frac {\varepsilon}{\Delta} f _ {i} \left(x ^ {\prime}\right)\right)} \tag {2}
$$
First we let  $\Delta_1' = \max\{0, \max_{\ell \leq k} (f_\ell(x') - f_\ell(x))\}$  and  $\Delta_2' = f_k(x) - f_k(x')$ , and we will achieve our desired inequality (2) by bounding the numerator with respect to  $\Delta_2'$  and the denominator with respect to  $\Delta_1'$ . We have  $\exp(\frac{\epsilon}{\Delta} f_k(x)) = \exp(\epsilon \Delta_2' / \Delta) \exp(\frac{\epsilon}{\Delta} f_k(x'))$  by definition of  $\Delta_2'$ . Further  $\exp(\frac{\epsilon}{\Delta} f_i(x')) \leq \exp(\epsilon \Delta_1' / \Delta) \exp(\frac{\epsilon}{\Delta} f_i(x))$  by definition of  $\Delta_1'$ , so
$$
\exp \left(\frac {\epsilon}{\Delta} T\right) + \sum_ {i = 1} ^ {k} \exp \left(\frac {\epsilon}{\Delta} f _ {i} \left(x ^ {\prime}\right)\right) \leq e ^ {\frac {\epsilon \Delta_ {1} ^ {\prime}}{\Delta}} \left(\exp \left(\frac {\epsilon}{\Delta} T\right) + \sum_ {i = 1} ^ {k} \exp \left(\frac {\epsilon}{\Delta} f _ {i} (x)\right)\right)
$$
because  $\epsilon \Delta_1' / \Delta \geq 0$ . In order to show our desired inequality (2) it then suffices to show  $\Delta_2 \geq \Delta_1' + \Delta_2'$ . By definition we know  $\Delta_2 \geq \Delta_1 + \Delta_2'$ . Furthermore, if  $\Delta_1' > \Delta_1$  then  $\Delta_1' = f_k(x') - f_k(x) = -\Delta_2'$  which implies  $\Delta_1' + \Delta_2' = 0$ , and we know  $\Delta_2 \geq 0$  by construction.
Combining inequality (1) and (2) along with the fact that  $\Delta_1 = \Delta_k(x,x')$  from Definition B.1 completes the proof.
Now that we've bounded the one-sided privacy loss for neighboring datasets for each type of noise, this will immediately imply an overall privacy bound over all neighbors that utilizes this general characterization.
Corollary B.1. For a stream of queries  $\{f_i\}$  with sensitivity  $\Delta$  and threshold  $T$ , Algorithm 1 with Lap, Gumbel, or Expo noise is  $\varepsilon$ -DP where
$$
\varepsilon = \max  _ {x \sim x ^ {\prime}} \varepsilon \left(x, x ^ {\prime}\right)
$$
and for Gumbel noise we must have  $\varepsilon_1 = \varepsilon_2$ .
B.2 Range-bounded definition and properties
The definition of range-bounded was originally introduced in Durfee and Rogers (2019) to tighten the privacy bounds on the composition of exponential mechanisms. The analysis was further extended in Dong et al. (2020) to give the optimal composition of range-bounded mechanisms. This definition was then unified with other definitions in Cesar and Rogers (2021).
Definition B.2 (Durfee and Rogers (2019)). A mechanism  $M: \mathcal{X} \to \mathcal{Y}$  is  $\varepsilon$ -range-bounded if for any neighboring datasets  $x, x' \in \mathcal{X}$  and outcomes  $y, y' \in \mathcal{Y}$ :
$$
\frac {P r [ M (x) = y ]}{P r [ M \left(x ^ {\prime}\right) = y ]} \leq e ^ {\varepsilon} \frac {P r [ M (x) = y ^ {\prime} ]}{P r [ M \left(x ^ {\prime}\right) = y ^ {\prime} ]}
$$
If we have bounds upon this property that are stronger than those immediately implied by DP, then it was shown in Dong et al. (2020) that substantial improvements can be made in bounding the overall DP over the composition of these mechanisms. While these improvements can be applied to our results here as well, we focus upon the privacy bounds with respect to zCDP as it is much cleaner to work with in providing strong composition bounds. From the proof of Lemma 3.4 in Cesar and Rogers (2021) we see that the range-bounded property can give a significant improvement in zCDP properties compared to similar DP guarantees.
Proposition 2 (Cesar and Rogers (2021)). If a mechanism  $M$  is  $\varepsilon$ -range-bounded then it is  $\frac{1}{8} \varepsilon^2 - zCDP$ .
Our general characterization is already set up to provide range-bounded properties of AboveThreshold. This will allow us to give tighter guarantees on the range-boundedness which can be translated to better privacy composition bounds.
Corollary B.2. For a stream of queries  $\{f_i\}$  and threshold  $T$ , Algorithm 1 with Lap, Gumbel, or Expon noise is  $\varepsilon$ -range-bounded where
$$
\varepsilon = \max  _ {x \sim x ^ {\prime}} \left(\varepsilon \left(x, x ^ {\prime}\right) + \varepsilon \left(x ^ {\prime}, x\right)\right)
$$
Proof. We can rewrite the inequality from Definition B.2 as equivalently
$$
\Pr \left[ M (x) = y \right] \cdot \Pr \left[ M \left(x ^ {\prime}\right) = y ^ {\prime} \right] \leq e ^ {\varepsilon} \Pr \left[ M \left(x ^ {\prime}\right) = y \right] \cdot \Pr \left[ M (x) = y ^ {\prime} \right]
$$
Our claim then follows immediately by applying Lemma B.1 and Lemma B.3
B.3 Improved composition analysis of SVT
In this section we complete our analysis for improving the privacy bounds for sparse vector technique. While our generalized characterization was in a more complex form, we simplify it here for general queries and monotonic queries. This will then match up to the privacy guarantees that we gave in Section 3 for AboveThreshold.
Lemma B.4. Given a stream of queries  $\{f_i\}$  with sensitivity  $\Delta$  then for any neighboring datasets  $\varepsilon(x, x') \leq \varepsilon_1 + 2\varepsilon_2$ . If the queries are monotonic then for any neighboring datasets  $\varepsilon(x, x') \leq \varepsilon_1 + \varepsilon_2$ , and furthermore,  $\varepsilon(x, x') + \varepsilon(x', x) \leq \varepsilon_1 + 2\varepsilon_2$ .
Proof. By Definition 2.2 we must always have  $\Delta_k(x, x') \leq \Delta$  for any neighboring datasets. Furthermore, we must also have  $f_k(x') - f_k(x) \geq -\Delta$  for any neighboring datasets. This implies  $\varepsilon(x, x') \leq \varepsilon_1 + 2\varepsilon_2$  for any neighboring datasets.
Now assume the queries are monotonic. Without loss of generality assume  $f_{k}(x) \geq f_{k}(x^{\prime})$  for all  $k$ . Therefore  $\Delta_k(x, x') = 0$  and  $f_{k}(x') - f_{k}(x) \geq -\Delta$ , which implies  $\varepsilon(x, x') \leq \varepsilon_2$ . Similarly we have  $\Delta_k(x', x) \leq \Delta$  but also  $f_{k}(x) - f_{k}(x') \geq 0$ , which implies  $\varepsilon(x', x) \leq \varepsilon_1 + \varepsilon_2$ . Combining these implies  $\varepsilon(x', x) \leq \varepsilon_1 + \varepsilon_2$  and  $\varepsilon(x', x) + \varepsilon(x, x') \leq \varepsilon_1 + 2\varepsilon_2$  for any neighboring datasets.
This lemma along with Corollary B.1 immediately imply Corollary 3.1 and Lemma 3.2
Proof of Theorem 1. From Lemma B.4 and Corollary B.2 we have that Algorithm 1 is  $\varepsilon_1 + 2\varepsilon_2$ -ranged-bounded. Applying Proposition 2 then gives the zCDP guarantees.
The sparse vector technique is simply an iterative call to AboveThreshold and as such the privacy bounds will come from the composition. By analyzing the composition through zCDP which has become commonplace in the privacy community, we can immediately improve the privacy guarantees for monotonic queries in the sparse vector technique. Furthermore we can also improve the privacy guarantees under the standard definition using the improved composition bounds for range-bounded mechanisms from Dong et al. (2020).
We also note here that the generalized Sparse Vector Technique given in Lyu et al. (2017) only adds noise once to the threshold and only has to scale the noise to the number of calls to AboveThreshold for the queries and not the threshold. Our analysis can extend to give the same properties for Lap and Expo noise, but utilizing advanced composition properties will give far more accuracy. More specifically, the threshold and queries having noise proportional to  $\sqrt{c} / \varepsilon$ , is preferable to all the queries having noise proportional to  $c / \varepsilon$ , and only the threshold proportional to  $1 / \varepsilon$ , where  $c$  is the number of calls to AboveThreshold.
B.4 Iterative exponential mechanism
It is folklore that the peeling exponential mechanism is equivalent to taking the top- $k$  after adding Gumbel noise, as formally shown in Lemma 4.2 of Durfee and Rogers (2019). We show a similar property here for AboveThreshold with Gumbel noise that it is equivalent to iteratively running the exponential mechanism.
Algorithm 3 Iterative Exponential Mechanism
Require: Input dataset  $x$ , a stream of queries  $\{f_i : \mathcal{X} \to \mathbb{R}\}$  with sensitivity  $\Delta$ , and a threshold  $T$   
1: for each query  $i$  do  
2: Run exponential mechanism with  $\mathcal{Y} = \{0, \dots, i\}$  where  $q(x, 0) = T$  and  $q(x, j) = f_j(x)$  for all  $j > 0$   
3: if  $i$  is selected then  
4: Output  $\top$  and halt  
5: else  
6: Output  $\bot$   
7: end if  
8: end for
Lemma B.5. If  $\varepsilon_1 = \varepsilon_2 = \varepsilon / 2$  then Algorithm 1 with Gumbel noise gives the equivalent output distribution to Algorithm 3.
Proof. We first define
$$
p _ {k} = \frac {\exp \left(\frac {\varepsilon}{2 \Delta} f _ {k} (x)\right)}{\exp \left(\frac {\varepsilon}{2 \Delta} T\right) + \sum_ {i = 1} ^ {k} \exp \left(\frac {\varepsilon}{2 \Delta} f _ {i} (x)\right)}
$$
By construction, the probability of Algorithm 3 outputting  $\{\bot^{k - 1},\top \}$  is equal to  $p_k\prod_{i = 1}^{k - 1}(1 - p_i)$ . Furthermore, by our definition of  $p_k$  we have
$$
1 - p _ {k} = \frac {\exp \left(\frac {\varepsilon}{2 \Delta} T\right) + \sum_ {i = 1} ^ {k - 1} \exp \left(\frac {\varepsilon}{2 \Delta} f _ {i} (x)\right)}{\exp \left(\frac {\varepsilon}{2 \Delta} T\right) + \sum_ {i = 1} ^ {k} \exp \left(\frac {\varepsilon}{2 \Delta} f _ {i} (x)\right)}
$$
Through telescoping cancellation
$$
\prod_ {i = 1} ^ {k - 1} \left(1 - p _ {i}\right) = \frac {\exp \left(\frac {\varepsilon}{2 \Delta} T\right)}{\exp \left(\frac {\varepsilon}{2 \Delta} T\right) + \sum_ {i = 1} ^ {k - 1} \exp \left(\frac {\varepsilon}{2 \Delta} f _ {i} (x)\right)}
$$
From Lemma B.2 we then see that the output probabilities are equivalent.
C Additional unbounded quantile estimation results
In this section we first extend our method to the fully unbounded setting by simply making two calls to AboveThreshold and essentially searching through both positive and negative numbers in each respective call. Next we show how our methods can also extend to the add-subtract definition of neighboring. Further, we'll apply our approach to the framework set up in Kaplan et al. (2022) for computing multiple quantiles and extend it to the swap definition.
C.1 Fully unbounded quantile estimation
Recall that our unbounded quantile estimation algorithm assumed that the data was lower bounded. It then slowly increased that bound by a small percentage until the appropriate amount of data fell below the threshold for the given quantile. In order to extend to the fully unbounded setting, we will simply first apply this guess and check method to the positive numbers and then apply it to the negative numbers.
Algorithm 4 Fully unbounded quantile mechanism
Require: Input dataset  $x$ , a quantile  $q$ , and parameter  $\beta > 1$
1: Run AboveThreshold with  $x, T = qn$  and  $f_{i}(x) = |\{x_{j} \in x | x_{j} + 1 < \beta^{i}\}|$
2: Run AboveThreshold with  $x, T = (1 - q)n$  and  $f_{i}(x) = |\{x_{j} \in x | x_{j} - 1 > -\beta^{i}\}|$
3: If the first AboveThreshold halts at  $k > 0$  then output  $\beta^k - 1$
4: If the second AboveThreshold halts at  $k > 0$  then output  $-\beta^k + 1$
5: Otherwise return 0
Note that the second call could equivalently be achieved by flipping the sign of all the datapoints and again applying queries  $f_{i}(x) = |\{x_{j}\in x|x_{j} + 1 < \beta^{i}\} |$ . Therefore all our privacy guarantees from Section 4 will still apply, but composing over two calls to AboveThreshold, which is the Sparse Vector Technique.
In the first call to AboveThreshold we are assuming a lower bound of 0, and searching the positive numbers. If it terminates immediately then it is likely that more than a qth fraction of the data is below 0. We then symmetrically search through the negative numbers by assuming an upper bound of 0. If this halts immediately then it is likely that the quantile is already near 0. We could also apply other variants of this, such as three total calls to get the maximum and minimum of the data if we wanted full data bounds.
Further note that computing the smallest quantiles will be challenging for our algorithm because it may take many queries to get to the appropriate threshold, but each query will have a reasonable chance of terminating if  $q$  is very small. To account for these queries in the lower-bounded setting, we can invert all the datapoints and instead search for quantile  $1 - q$  on this transformed data, then invert our resulting estimate. We would also need to reduce our parameter  $\beta$  a reasonable amount in this setting.
C.2 Extension to add-subtract neighbors
We also extend our results to the add-subtract definition of neighboring datasets.
Definition C.1. Datasets  $x, x' \in \mathcal{X}$  are neighboring if one of them can be obtained from the other by adding or removing one individual's data.
Under this definition we see that our threshold  $T = qn$  is no longer fixed, so we will instead set each query to be  $f_{i}(x) = |\{x_{j} \in x | x_{j} - \ell + 1 < \beta^{i}\}| - qn$ . Unfortunately this query will no longer be monotonic, so we will instead take advantage of our more general characterization in Section B.1 to further tighten the privacy bounds in this setting.
Lemma C.1. Given the stream of queries  $f_{i}(x) = |\{x_{j}\in x|x_{j} - \ell +1 < \beta^{i}\} | - qn$  with sensitivity  $\Delta = 1$ , for any neighboring datasets  $x,x^{\prime}$  under Definition C.1 and quantile  $q\in [0,1]$  we have  $\varepsilon (x,x^{\prime})\leq \max \{(1 - q)\varepsilon_{1},q\varepsilon_{1} + \varepsilon_{2}\}$ .
Proof. Without loss of generality, assume that  $x$  has one more individuals data, so  $x \in \mathbb{R}^n$  and  $x' \in \mathbb{R}^{n-1}$ . Let  $g_i(x) = |\{x_j \in x | x_j - \ell + 1 < \beta^i\}|$ . By construction we must have  $g_i(x) - g_i(x') \in \{0, 1\}$  because  $x$  has an additional datapoint. Furthermore, because the thresholds are increasing, if  $g_k(x) - g_k(x') = 1$  then  $g_i(x) - g_i(x') = 1$  for all  $i > k$ . Similarly if  $g_k(x) - g_k(x') = 0$  then  $g_i(x) - g_i(x') = 0$  for all  $i < k$ . We further see that  $f_i(x) - f_i(x') = g_i(x) - g_i(x') - q$  for all  $i$ .
First consider the case when  $g_{k}(x) - g_{k}(x^{\prime}) = 0$ . We therefore have  $g_{i}(x) - g_{i}(x^{\prime}) = 0$  for all  $i < k$  so  $\Delta_k(x,x') = q$ , and also  $\Delta_k(x,x') - (f_k(x') - f_k(x)) = 0$ , so  $\varepsilon (x,x^{\prime})\leq q\varepsilon_{1}$ . Similarly, we have  $\Delta_k(x',x) = 0$  and  $\Delta_k(x,x') - (f_k(x') - f_k(x)) = q$ , so  $\varepsilon (x',x)\leq q\varepsilon_2$ .