SlowGuess/NeurIPS_2023_OCR huggingface dataset:
[1] Hilal Asi and John C. Duchi. Instance-optimality in differential privacy via approximate inverse sensitivity mechanisms. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS'20, Red Hook, NY, USA, 2020. Curran Associates Inc.
[2] Grey Ballard, Tamara G Kolda, Ali Pinar, and C Seshadhri. Diamond sampling for approximate maximum all-pairs dot-product (mad) search. In 2015 IEEE International Conference on Data Mining, pages 11-20. IEEE, 2015.
[3] Jeremiah Blocki, Avrim Blum, Anupam Datta, and Or Sheffet. Differentially private data analysis of social networks via restricted sensitivity. In Proceedings of the 4th Conference on Innovations in Theoretical Computer Science, ITCS '13, page 87-96, New York, NY, USA, 2013. Association for Computing Machinery.
[4] Jeremiah Blocki, Elena Grigorescu, Tamalika Mukherjee, and Samson Zhou. How to make your approximation algorithm private: A black-box differentially-private transformation for tunable approximation algorithms of functions with low sensitivity. arXiv preprint arXiv:2210.03831, 2022.
[5] Marco Bressan. Faster subgraph counting in sparse graphs. In 14th International Symposium on Parameterized and Exact Computation (IPEC 2019). Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik, 2019.
[6] Marco Bressan and Marc Roth. Exact and approximate pattern counting in degenerate graphs: New algorithms, hardness results, and complexity dichotomies. In 2021 IEEE 62nd Annual Symposium on Foundations of Computer Science (FOCS), pages 276-285. IEEE, 2022.
[7] Shixi Chen and Shuigeng Zhou. Recursive mechanism: Towards node differential privacy and unrestricted joins. In Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data, SIGMOD '13, page 653-664, New York, NY, USA, 2013. Association for Computing Machinery.
[8] Vincent Cohen-Addad, Chenglin Fan, Silvio Lattanzi, Slobodan Mitrovic, Ashkan Norouzi-Fard, Nikos Parotsidis, and Jakub M Tarnawski. Near-optimal correlation clustering with privacy. Advances in Neural Information Processing Systems, 35:33702-33715, 2022.
[9] Laxman Dhulipala, Quanquan C Liu, Sofya Raskhodnikova, Jessica Shi, Julian Shun, and Shangdi Yu. Differential privacy from locally adjustable graph algorithms: k-core decomposition, low out-degree ordering, and densest subgraphs. In 2022 IEEE 63rd Annual Symposium on Foundations of Computer Science (FOCS), pages 754-765. IEEE, 2022.
[10] Xiaofeng Ding, Xiaodong Zhang, Zhifeng Bao, and Hai Jin. Privacy-preserving triangle counting in large graphs. In Proceedings of the 27th ACM international conference on information and knowledge management, pages 1283-1292, 2018.
[11] Rodney G Downey and Michael R Fellows. Fundamentals of parameterized complexity, volume 4. Springer, 2013.
[12] Cynthia Dwork, Aaron Roth, et al. The algorithmic foundations of differential privacy. Foundations and Trends® in Theoretical Computer Science, 9(3-4):211-407, 2014.
[13] David Eppstein, Maarten Löffler, and Darren Strash. Listing all maximal cliques in sparse graphs in near-optimal time. arXiv preprint arXiv:1006.5440, 2010.
[32] Davide Proserpio, Sharon Goldberg, and Frank McSherry. Calibrating data to sensitivity in private data analysis. Proceedings of the VLDB Endowment, 7(8), 2014.
[33] Zhan Qin, Ting Yu, Yin Yang, Issa Khalil, Xiaokui Xiao, and Kui Ren. Generating synthetic decentralized social graphs with local differential privacy. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (CCS), pages 425-438, 2017.
[34] Vibhor Rastogi, Michael Hay, Gerome Miklau, and Dan Suciu. Relationship privacy: Output perturbation for queries with joins. In Proceedings of the Twenty-Eighth ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems, PODS '09, page 107-116, New York, NY, USA, 2009. Association for Computing Machinery.
[35] Jessica Shi, Laxman Dhulipala, and Julian Shun. Parallel clique counting and peeling algorithms. In SIAM Conference on Applied and Computational Discrete Algorithms (ACDA21), pages 135-146. SIAM, 2021.
[36] Jun Zhang, Graham Cormode, Cecilia M. Procopiuc, Divesh Srivastava, and Xiaokui Xiao. Private release of graph statistics using ladder functions. In Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data, SIGMOD '15, page 731-745, New York, NY, USA, 2015. Association for Computing Machinery.
A Differential Privacy with high probability
Following [24], we define Indistinguishability and Point-wise Indistinguishability as follows:
Definition 8. Two random variables  $X, Y$  having the same support are  $(\epsilon, \delta)$ -indistinguishable if for all set  $O \subseteq \operatorname{Supp}(X)$ ,
$$
\Pr [ X \in O ] \leq e ^ {\epsilon} \Pr [ Y \in O ] + \delta a n d \Pr [ Y \in O ] \leq e ^ {\epsilon} \Pr [ Y \in O ] + \delta
$$
Definition 9. Two random variables  $X, Y$  are point-wise  $(\epsilon, \delta)$ -indistinguishable if with probability at least  $1 - \delta$  over a drawn from either  $X$  or  $Y$ , we have:
$$
e ^ {- \epsilon} \Pr [ Y = a ] \leq \Pr [ X = a ] \leq e ^ {\epsilon} \Pr [ Y = a ]
$$
The following lemma allows us to interpret a mechanism that is  $\epsilon$ -differentially private with probability at least  $1 - \delta$  as  $(\epsilon, \delta)$ -differentially private.
Lemma 7. (Lemma 3.3 of [24]) If  $X, Y$  are point-wise  $(\epsilon, \delta)$ -indistinguishable then they are  $(\epsilon, \delta)$ -indistinguishable.
We extend the definition of  $(\epsilon, \delta)$ -indistinguishability by allowing some small error probability  $\delta'$ :
Definition 10. Two random variables  $X, Y$  having the same Support are  $(\epsilon, \delta, \delta')$ -indistinguishable if for all set  $O \subseteq \operatorname{Supp}(X)$ , with probability at least  $1 - \delta'$  over a drawn from either  $X$  or  $Y$ , we have:
The following lemma allows us to interpret a mechanism that is  $(\epsilon, \delta)$ -differentially private with probability at least  $1 - \delta'$  as  $(\epsilon, \delta + \delta')$ -differentially private.
Lemma 8. If  $X, Y$  are  $(\epsilon, \delta, \delta')$ -indistinguishability then they are  $(\epsilon, \delta + \delta')$ -indistinguishable.
Proof. Our proof is inspired by [24].
We define the set  $Bad$  as follow:
$$
B a d = \left\{O: \Pr [ X \in O ] > e ^ {\epsilon} \Pr [ Y \in O ] + \delta \text {o r} \Pr [ Y \in O ] > e ^ {\epsilon} \Pr [ Y \in O ] + \delta \right\}
$$
Since  $X, Y$  are  $(\epsilon, \delta, \delta')$ -indistinguishability,  $\operatorname{Pr}[X \in Bad] < \delta'$ . We have:
$$
\begin{array}{l} \Pr [ X \in O ] \leq \Pr [ X \in O \backslash B a d ] + \Pr [ X \in B a d ] \\ \leq e ^ {\epsilon} \Pr \left[ Y \in O \backslash B a d \right] + \delta + \delta^ {\prime} \\ \leq e ^ {\epsilon} \Pr [ Y \in O ] + \delta + \delta^ {\prime}. \\ \end{array}
$$
Similarly for  $Y$ , we have  $\operatorname*{Pr}[Y \in O] \leq e^{\epsilon} \operatorname*{Pr}[X \in O] + \delta + \delta'$  and the Lemma follows.
Approximate Smooth Sensitivity for exact queries
When we can calculate a  $\gamma$ -approximation of the smooth sensitivity  $S_{f,\beta}^{*}$ , i.e.,  $\tilde{S}_{f,\beta}$  is a bounded approximation of  $S_{f,\beta}^{*}$  with probability 1, we can substitute  $S_{f,\beta}^{*}$  by  $\tilde{S}_{f,\beta}$  to calibrate the noise directly, although with a bit larger effective constant  $\beta' = \beta + \gamma$ , which in turn will make the noise a bit larger. Lemma 9 formalizes the equivalence of  $S_{f,\beta}^{*}$  and  $\tilde{S}_{f,\beta}$  in this case.
Lemma 9. If  $\tilde{S}_{f,\beta}$  is a  $\gamma$ -approximation of the smooth sensitivity  $S_{f,\beta}^{*}$ , then  $\tilde{S}_{f,\beta}$  is a  $\beta'$ -smooth sensitivity of  $f$  at  $\beta' = \beta + \gamma$ .
Proof. By definition of  $\tilde{S}_{f,\beta}$  and  $S_{f,\beta}^{*}$ , it follows that  $\tilde{S}_{f,\beta}(D) \geq S_{f,\beta}^{*}(D) \geq LS_{f}(D)$  for all  $x$ .
Second, for all  $D \sim D' \in D^n$ , we have:  $\tilde{S}_{f,\beta}(D) \leq e^{\gamma} S_{f,\beta}^{*}(D) \leq e^{\gamma} e^{\beta} S_{f,\beta}^{*}(D') \leq e^{\gamma + \beta} \tilde{S}_{f,\beta}(D') \leq e^{\beta'} \tilde{S}_{f,\beta}(D')$ , the lemma follows from Definition 2.
In the case when we can only calculate a  $(\gamma, \delta')$ -approximation of the smooth sensitivity, i.e.,  $\tilde{S}_{f,\beta}$  is a  $\gamma$ -approximation of  $S_{f,\beta}^{*}$  with probability at least  $1 - \delta'$ , Lemma 9 cannot be directly applied. Similar to [31], we use the formalism of admissible noise, to prove we can use  $\tilde{S}_{f,\beta}$  to calibrate an  $(\alpha, \beta)$ -admissible noise with appropriate values of  $\alpha$  and  $\beta$  to deliver the privacy guarantee. In Lemma 10, we analyze the output of such a mechanism on an arbitrary pair of neighbors  $D$  and  $D'$ . We condition our analysis on the events of success of  $\tilde{S}_{f,\beta}$  on both  $D$  and  $D'$  (with probability at least  $1 - 2\delta'$ ). The failure probability (at most  $2\delta'$ ) will then add up to the  $\delta$ -part of the final  $(\epsilon, \delta)$ -DP guarantee.
Lemma 10. Let  $h$  be  $(\alpha, \beta')$ -admissible noise. Let  $\tilde{S}_{f,\beta}$  be a  $(\gamma, \delta')$ -approximation to  $S_{f,\beta}^*$ , where  $\beta = \beta' - \gamma$ . Then,  $A(D) = f(D) + \frac{\tilde{S}_{f,\beta}(D)}{\alpha} Z$  is  $(\epsilon, \frac{e^{\epsilon/2} + 1}{2}\delta + 2\delta')$ -differentially private, where  $Z \sim h$ .
Proof. We prove that with probability at least  $1 - 2\delta'$ ,  $A(D)$  is  $(\epsilon, \frac{e^{\epsilon/2} + 1}{2}\delta)$ -differentially private, and by applying Lemma 8 the Lemma follows.
Let  $\mathcal{E}$  be the event that  $S^{*}(D)\leq \tilde{S}_{f,\beta}(D)\leq e^{\gamma}S^{*}(D)$ ; we have  $\operatorname *{Pr}[\mathcal{E}]\geq 1 - \delta '$ . Similarly, let  $\mathcal{E}'$  be the event that  $S^{*}(D^{\prime})\leq \tilde{S}_{f,\beta}(D^{\prime})\leq e^{\gamma}S^{*}(D^{\prime})$  for a fixed neighbor  $D^{\prime}$  of  $D$ ; we have  $\operatorname *{Pr}[\mathcal{E}']\geq 1 - \delta ',$  and  $\operatorname *{Pr}[\mathcal{E}\cup \mathcal{E}']\geq 1 - 2\delta '.$  We will condition on  $\mathcal{E}\cup \mathcal{E}'$
Let  $\mathcal{S} \subset \mathbb{R}^+$ . Let  $N(D) = \frac{\tilde{S}_{f,\beta}(D)}{\alpha}$ , and define  $\mathcal{S}_1 = \frac{\mathcal{S} - f(D)}{N(D)}, \mathcal{S}_2 = \mathcal{S}_1 + \frac{f(D) - f(D')}{N(D)} = \frac{\mathcal{S} - f(D')}{N(D)}$ ,  $\mathcal{S}_3 = \mathcal{S}_2 \frac{N(D)}{N(D')} = \frac{\mathcal{S} - f(D')}{N(D')}$ .
We observe that  $A(D) \in S$  if and only if  $Z \in S_1$ . Similarly,  $A(D') \in S$  if and only if  $Z \in S_3$ . We have
$$
\begin{array}{l} \Pr [ A (D) \in \mathcal {S} ] = \Pr [ Z \in \mathcal {S} _ {1} ] \leq e ^ {\epsilon / 2} \Pr \left[ Z \in \mathcal {S} _ {1} + \frac {f (D) - f \left(D ^ {\prime}\right)}{N (D)} \right] + \delta / 2 \\ = e ^ {\epsilon / 2} \operatorname * {P r} [ Z \in \mathcal {S} _ {2} ] + \delta / 2 \leq e ^ {\epsilon} \operatorname * {P r} \left[ Z \in \mathcal {S} _ {2} \exp \left(\ln \frac {N (D)}{N (D ^ {\prime})}\right) \right] + e ^ {\epsilon / 2} \delta / 2 + \delta / 2 \\ = e ^ {\epsilon} \Pr [ Z \in \mathcal {S} _ {3} ] + e ^ {\epsilon / 2} \delta / 2 + \delta / 2 = e ^ {\epsilon} \Pr [ A (D ^ {\prime}) \in \mathcal {S} ] + (e ^ {\epsilon / 2} + 1) \delta / 2, \\ \end{array}
$$
where the first inequality is because of Sliding Property and
$$
\frac {| f (D) - f (D ^ {\prime}) |}{N (D)} = \alpha \frac {| f (D) - f (D ^ {\prime}) |}{\tilde {S} _ {f , \beta} (D)} \leq \alpha \frac {| f (D) - f (D ^ {\prime}) |}{S ^ {*} (D)} \leq \alpha ,
$$
which satisfies the condition for the Sliding Property,
and the second inequality is because of Dilation Property and
$$
\left| \ln \frac {N (D)}{N (D ^ {\prime})} \right| = \left| \ln \frac {\tilde {S} _ {f , \beta} (D)}{\tilde {S} _ {f , \beta} (D ^ {\prime})} \right| \leq \left| \ln \left(e ^ {\gamma} \frac {S ^ {*} (D)}{S ^ {*} (D ^ {\prime})}\right) \right| \leq | \gamma + \ln e ^ {\pm \beta} | \leq \beta^ {\prime},
$$
which satisfies the condition for the Dilation Property.
Lemma 11 specializes Lemma 10 by using Laplacian noise. We note that this is a generalization of [31]'s analysis of Laplacian noise. We introduce parameter  $k$  to control the magnitude of the Laplacian noise, while [31]'s analysis fixed it to 1. This design gives us the flexibility to choose the values of  $\alpha$  and  $\beta$  given fixed targets of  $\epsilon$  and  $\delta$ . When  $k = 1/2$ , our design yields the same results as [31].
Lemma 11. Let  $\alpha = k\epsilon$  and  $\beta = \frac{2k^2\epsilon}{\log(2 / \delta)}$  for any constant  $k > 0$ , then  $Lap(2k)$  is an  $(\alpha, \beta)$ -admissible noise.
Proof. For som real-valued random variable  $Y$ , let  $\rho_{\delta}(Y)$  be the least solution to  $\operatorname*{Pr}[Y \leq \rho_{\delta}] > 1 - \delta$ . Let  $\delta' = \delta / 2$ . Let  $Z \sim \text{Lap}(2k)$ . It follows that  $|Z| \sim \text{Exponential}(1/(2k))$ . We will solve the value of  $\rho_{\delta'}(|Z|)$  as:  $1 - e^{-2k|\rho_{\delta'}(|Z|)|} = 1 - \delta'$  or  $\rho_{\delta'}(|Z|) = \frac{\log(2/\delta)}{2k}$ .
Substituting  $\rho_{\delta'}(|Z|)$  to the formula of  $\beta$ , we have:
$$
\begin{array}{l} \beta = \frac {2 k ^ {2} \epsilon}{\log (2 / \delta)} \\ = \frac {k \epsilon}{\rho_ {\delta^ {\prime}} (| Z |)} \\ \end{array}
$$
We first prove the Sliding property of  $Lap(2k)$ . It is equivalent to prove that  $\ln \left(\frac{h(z + \Delta)}{h(z)}\right) \leq \epsilon / 2$  for any  $|\Delta| < \alpha$  and  $h(z)$  is the PDF of  $Lap(2k)$ . For the distribution  $Lap(2k)$ ,  $h(z) = \frac{1}{4k} e^{-|z| / (2k)}$ . We have:
$$
\begin{array}{l} \ln \left(\frac {h (z + \Delta)}{h (z)}\right) = \ln \left(\frac {\exp (- | z + \Delta | / (2 k))}{\exp (- | z | / (2 k))}\right) \\ = \frac {1}{2 k} (| z | - | z + \Delta |) \\ \leq \frac {1}{2 k} | \Delta | \\ \leq \frac {1}{2 k} k \epsilon \\ = \frac {\epsilon}{2}. \\ \end{array}
$$
We next prove the Dilation property of  $Lap(2k)$ , or prove that  $\ln \left(\frac{e^{\lambda}h(e^{\lambda}z)}{h(z)}\right) \leq \epsilon / 2$  with probability at least  $1 - \delta / 2$ . In the first case, setting  $\lambda > 0$ , we have  $h(e^{\lambda}z) < h(z)$ , and that  $\ln \left(\frac{e^{\lambda}h(e^{\lambda}z)}{h(z)}\right) \leq \lambda \leq \frac{k\epsilon}{\rho_{\delta'}(|Z|)}$ . Since  $|Z| \sim \text{Exponential}(1/(2k))$ , the median of the distribution of  $|Z|$  is  $\frac{\ln(2)}{1/(2k)} = 2\ln(2)k$ . With  $\delta < 1$ , we have  $\delta' < 1/2$  and therefore  $\rho_{\delta'}(|Z|) \geq \text{median}(|Z|) = 2\ln(2)k$ . It follows that  $\ln \left(\frac{e^{\lambda}h(e^{\lambda}z)}{h(z)}\right) \leq \frac{k\epsilon}{2\ln(2)k} < \frac{\epsilon}{2}$ .
In the second case,  $\lambda < 0$ . We consider the ratio  $\frac{h(e^{\lambda}z)}{h(z)}$ :
$$
\begin{array}{l} \frac {h (e ^ {\lambda} z)}{h (z)} = \frac {\exp (- | z e ^ {\lambda} | / (2 k))}{\exp (- | z | / (2 k))} \\ = \exp \left(\frac {| z |}{2 k} (1 - e ^ {\lambda})\right) \\ \leq \exp \left(\frac {| z |}{2 k} | \lambda |\right). \\ \end{array}
$$
We then have  $\ln \left(\frac{e^{\lambda}h(e^{\lambda}z)}{h(z)}\right) \leq \lambda + \frac{|z||\lambda|}{2k} < \frac{|z||\lambda|}{2k}$ . Consider the event  $G: \{z: |z| < \rho_{\delta'}(|Z|)\}$ . Under  $G$ , we have  $|z||\lambda| \frac{1}{2k} \leq \rho_{\delta'}(|Z|) \frac{\epsilon}{\rho_{\delta'}(|Z|)} \frac{1}{2k} = \frac{\epsilon}{2}$ . Under  $h$ ,  $\operatorname*{Pr}[G] = 1 - \delta / 2$ , which completes the proof for the Dilation property of  $Lap(2k)$ .
Finally, we apply a Laplacian noise with appropriate parameters to provide a concrete mechanism using Approximate Smooth Sensitivity  $\tilde{S}_{f,\beta}$  to guarantee  $(\epsilon ,\delta)$ -DP.
Lemma 12. (Lemma 2) Let  $\tilde{S}_{f,\beta}$  be a  $(\gamma, \delta')$ -approximation to  $S_{f,\beta}^{*}$ , for  $\gamma = \beta = \frac{16\epsilon}{\ln(2/\delta)}$ . Then,  $A(D) = f(D) + \operatorname{Lap}\left(\frac{2\tilde{S}_{f,\beta}(D)}{\epsilon}\right)$  is  $(\epsilon, \frac{e^{\epsilon/2} + 1}{2}\delta + 2\delta')$ -differentially private.
Proof. Set  $k = 4$ , using Lemma 11 with  $\alpha = k\epsilon = 4\epsilon$ , and  $\beta' = \beta + \gamma = \frac{32\epsilon}{\ln(2/\delta)} = \frac{2k^2\epsilon}{\ln(2/\delta)}$ . Then, from Lemma 11, the  $Lap(2k) = Lap(8)$  distribution is  $(\alpha, \beta')$ -admissible. By Lemma 10,  $A(D) = f(D) + \frac{\tilde{S}_{f,\beta}(D)}{\alpha} Z = f(D) + \frac{\tilde{S}_{f,\beta}(D)}{4\epsilon} Lap(8) = f(D) + Lap(2\tilde{S}_{f,\beta}(D)/\epsilon)$  is  $(\epsilon, \frac{e^{\epsilon/2} + 1}{2}\delta + 2\delta')$ -differentially private and the corollary follows.
Approximate Smooth Sensitivity for Approximate queries
Definition 11. (Definition 7) We define a function  $g_{f}$  and  $s_f$  as:
$$
g _ {f} = \left\{ \begin{array}{l} A _ {f} (D) \text {i f} (1 - \alpha_ {1}) f (D) \leq A _ {f} (D) \leq (1 + \alpha_ {1}) f (D), \\ (1 - \alpha_ {1}) f (D) \text {i f} A _ {f} (D) <   (1 - \alpha_ {1}) f (D), \\ (1 + \alpha_ {1}) f (D) \text {i f} A _ {f} (D) > (1 + \alpha_ {1}) f (D). \end{array} \right. \tag {3}
$$
$$
s _ {f} = \left\{ \begin{array}{l} \tilde {S} _ {f, \beta} (D) \text {i f} S _ {f, \beta} (D) \leq \tilde {S} _ {f, \beta} (D) \leq e ^ {\gamma} S _ {f, \beta} (D), \\ S _ {f, \beta} (D) \text {i f} \tilde {S} _ {f, \beta} (D) <   S _ {f, \beta} (D), \\ e ^ {\gamma} S _ {f, \beta} (D) \text {i f} \tilde {S} _ {f, \beta} (D) > e ^ {\gamma} S _ {f, \beta} (D). \end{array} \right. \tag {4}
$$
Let  $S_{g_f}(D) = 4\alpha_1 g_f(D) + 2 s_f(D)$
Algorithm 3 Fast algorithm using Approximate Smooth Sensitivity for Approximate Query  $A_{f}$  Input:  $G, f, \epsilon, \delta$
Output:  $(\epsilon, (e^{\epsilon/2} + 2)\delta)$ -differentially private estimation of query  $f$
1:  $\beta \coloneqq \gamma \coloneqq 4\alpha_{1} = \frac{\epsilon}{6\log(2 / \delta)}$
2:  $\delta_{1} := \delta_{2} = \delta / 2$
3: Calculate the  $(\gamma, \delta_2)$ -upper approximate smooth sensitivity  $\tilde{S}_{f,\beta}(G)$
4: Calculate the  $(\alpha_{1},\delta_{1})$  -approximate  $fA_{f}(G)$
5: Return  $A_{f}(G) + \frac{8\alpha_{1}A_{f}(G) + 4\tilde{S}_{f,\beta}(G)}{\epsilon} Lap(1)$
Lemma 13 proves that  $S_{g_f}$  is a  $\beta'$ -smooth upper bound of the local sensitivity of  $g_f$ , i.e., we have to prove the two conditions:
$LS_{g_f}(D)\leq Sg_f(D),$  and
-  $S_{g_f}(D) \leq e^{\beta'} S_{g_f}(D')$  for any pair of neighbor datasets  $D \sim D'$ .
where  $LS_{g_f}(D) = \max_{D' \sim D} |g_f(D) - g_f(D')|$  or the local sensitivity of  $g_f$  at  $D$ .
Lemma 13. (Lemma 3) Given  $g_{f}$ ,  $s_{f}$ , and  $S_{g_f}$  as defined above,  $S_{g_f}$  is a  $\beta^{\prime}$ -smooth upper bound of the local sensitivity of  $g_{f}$ , where  $\beta^{\prime}\geq 4\alpha_{1} + \gamma +\beta$  and  $\alpha_{1} < 1 / 2$ .
Proof. We now prove the two conditions. We first start with proving  $S_{g_f}$  is an upper bound on the local sensitivity  $LS_{g_f}$  of  $g_f$ :
$$
\begin{array}{l} L S _ {g _ {f}} (D) = \max  _ {D ^ {\prime}: D ^ {\prime} \sim D} \| g _ {f} (D) - g _ {f} \left(D ^ {\prime}\right) \| \\ \leq \max  _ {D ^ {\prime}: D ^ {\prime} \sim D} \| (1 + \alpha_ {1}) f (D) - (1 - \alpha_ {1}) f \left(D ^ {\prime}\right) \| \\ \leq \max  _ {D ^ {\prime}: D ^ {\prime} \sim D} \alpha_ {1} \| f (D) + f \left(D ^ {\prime}\right) \| + \| f (D) - f \left(D ^ {\prime}\right) \| \\ \leq \alpha_ {1} \| f (D) + f (D) + L S (D) \| + L S _ {f} (D) \\ \leq 2 \alpha_ {1} f (D) + 2 L S _ {f} (D) \\ \leq 4 \alpha_ {1} g _ {f} (D) + 2 S _ {f, \beta} (D) \\ \leq 4 \alpha_ {1} g _ {f} (D) + 2 s _ {f} (D) \\ = S _ {g _ {f}} (D). \\ \end{array}
$$
The first inequality uses Definition 7 and the fifth uses the facts,  $f(D) \leq \frac{A_f(D)}{1 - \alpha_1} \leq 2g_f(D)$ , and  $LS_{f}(D) \leq S_{f,\beta}(D)$  (as  $S_{f,\beta}(D)$  is a upper bound on  $LS_{f}(D)$ ).
For the second condition, we need to prove that for any pair  $D \sim D'$ , the values of  $S_{g_f}$  on them do not differ by more than a multiplicative factor of  $e^{\beta'}$  with  $\beta' = 4\alpha_1 + \gamma + \beta$ .
$$
\begin{array}{l} S _ {g _ {f}} (D) = 4 \alpha_ {1} g _ {f} (D) + 2 s _ {f} (D) \\ \leq 4 \alpha_ {1} (1 + \alpha_ {1}) f (D) + 2 s _ {f} (D) \\ \leq 4 \alpha_ {1} (1 + \alpha_ {1}) \left(f \left(D ^ {\prime}\right) + s _ {f} (D)\right) + 2 s _ {f} (D) \\ = 4 \alpha_ {1} (1 + \alpha_ {1}) f (D ^ {\prime}) + 2 (2 \alpha_ {1} (1 + \alpha_ {1}) + 1) s _ {f} (D) \\ \leq \frac {4 \alpha_ {1} (1 + \alpha_ {1})}{1 - \alpha_ {1}} g _ {f} \left(D ^ {\prime}\right) + 2 \left(2 \alpha_ {1} (1 + \alpha_ {1}) + 1\right) e ^ {\gamma + \beta} s _ {f} \left(D ^ {\prime}\right) \\ \leq 4 \alpha_ {1} (1 + \alpha_ {1}) (1 + 2 \alpha_ {1}) g _ {f} (D ^ {\prime}) + 2 (1 + 4 \alpha_ {1}) e ^ {\gamma + \beta} s _ {f} (D ^ {\prime}) \\ \leq 4 \alpha_ {1} \left(1 + 4 \alpha_ {1}\right) g _ {f} \left(D ^ {\prime}\right) + 2 \left(1 + 4 \alpha_ {1}\right) e ^ {\gamma + \beta} s _ {f} \left(D ^ {\prime}\right) \\ \leq e ^ {4 \alpha_ {1} + \gamma + \beta} \left(4 \alpha_ {1} g _ {f} \left(D ^ {\prime}\right) + 2 s _ {f} \left(D ^ {\prime}\right)\right) \\ = e ^ {\beta^ {\prime}} S _ {g _ {f}} (D ^ {\prime}). \\ \end{array}
$$
We note that  $s_f$ ,  $g_f$ , and  $S_{g_f}$  are all hypothetical functions, which we may not calculate directly. We need to connect the statement above ( $S_{g_f}$  is a Smooth Sensitivity of  $g_f$ ) to the functions we can compute ( $A_f$  and  $\tilde{S}_{f,\beta}$ ). Again, we define a hypothetical mechanism  $A'$  that takes  $D$  as its input. By the definition of  $A'$  and the properties of admissible noise,  $A'$  is  $(\epsilon, \delta)$ -DP. Instead of  $A'$ , we can only calculate  $A$ , by replacing  $g_f$  and  $s_f$  by  $A_f$  and  $\tilde{S}_{f,\beta}$  respectively. Next, we argue that  $A$  and  $A'$  are, in fact, not too different. Lemma 14 conditions on the probability that  $A$  and  $A'$  are the same and prove that the privacy guarantee of  $A'$  is the same with of  $A$  in such condition. The failure probabilities (of  $A_f$  and  $\tilde{S}_{f,\beta}$  in approximation of  $f$  and  $S_f$ ) will add up to the  $\delta$ -part of the final  $(\epsilon, \delta)$ -DP.
Lemma 14. Let  $h$  be  $(\alpha, \beta')$ -admissible noise with  $\beta' = 4\alpha_1 + \gamma + \beta$ . With  $A_f$  and  $\tilde{S}_{f,\beta}$  defined as above, Algorithm  $A(D) = A_f(D) + \frac{4\alpha_1A_f(D) + 2\tilde{S}_{f,\beta}(D)}{\alpha} Z$  is  $(\epsilon, \frac{e^{\epsilon/2} + 1}{2}\delta + 2\delta_1 + 2\delta_2)$ -differentially private, where  $Z \sim h$ .
Proof. We prove that with probability at least  $1 - 2\delta_{1} - 2\delta_{2}$ ,  $A(x)$  is  $(\epsilon, \frac{e^{\epsilon/2} + 1}{2}\delta)$ -differentially private, and by applying Lemma 8, the Lemma follows.
Let  $A^\prime (D) = g_f(D) + \frac{S_{g_f}}{\alpha} Z = g_f(D) + \frac{4\alpha_1s_f(D) + 2s_f(D)}{\alpha} Z$ , as we replace  $A_{f}$  and  $\tilde{S}_{f,\beta}$  by  $g_{f}$  and  $s_f$  respectively. We note that the use of  $g_{f}$  and  $s_f$  is only for the purpose of privacy bound analysis.
With  $\beta' > 4\alpha_{1} + \gamma + \beta$ , Lemma 3 shows that  $S_{g_f}$  is a  $\beta'$ -smooth upper bound on the sensitivity of  $g_{f}$ . By Lemma 10,  $A'(D) = g_{f}(D) + \frac{S_{g_{f}}}{\alpha} Z$  is  $(\epsilon, \frac{e^{\epsilon/2} + 1}{2}\delta)$ -differentially private.
Since with probability at least  $1 - \delta_{1}$ ,  $A_{f}(D)$  is  $g_{f}(D)$ , and at least  $1 - \delta_{2}$ ,  $\tilde{S}_{f,\beta}(D)$  is  $s_f(D)$ , with probability at least  $1 - \delta_1 - \delta_2$ ,  $A(D)$  has the same components as  $A^{\prime}(D)$ , hence they are identical. For a fixed neighbor  $D^{\prime} \sim D$ , we also have that  $A^{\prime}(D)$  is identical with  $A^{\prime}(D^{\prime})$  with probability at least  $1 - \delta_{1} - \delta_{2}$ . By Lemma 8,  $A(D), A(D^{\prime})$  are  $(\epsilon, \frac{e^{\epsilon/2} + 1}{2}\delta, 2\delta_{1} + 2\delta_{2})$ -indistinguishable. It follows that  $A(D)$  is  $(\epsilon, \frac{e^{\epsilon/2} + 1}{2}\delta + 2\delta_{1} + 2\delta_{2})$ -differentially private.