SlowGuess/NeurIPS_2023_OCR huggingface dataset:
The tree nodes at level  $\ell$  will partition the unit sphere  $S^{d - 1}$  into  $T^{\ell}$  regions, plus an extra "remainder" region of points that are not mapped into any of the  $T^{\ell}$  regions. At level 0, all points in  $S^{d - 1}$  are sent to the single region indexed by the root. For any node  $v$  in the tree of depth  $0\leq \ell \leq K - 1$  and with children  $v_{1},\ldots ,v_{K}$ , the region  $P_{v}$  is partitioned into  $P_{v_1},\dots,P_{v_T}$  as follows. A point  $p\in P_v$  is sent to  $P_{v_i}$  if  $i\le T$  is the smallest index such that  $\langle g_{v_i},p\rangle \geq \eta_u$ . Note that some points in  $P_{v}$  may not be sent to any of  $P_{v_1},\ldots ,P_{v_T}$ . For each leaf node  $v$ , we store an approximate count  $c_{v} = |X\cap P_{v}| + \mathrm{T}\mathrm{Lap}(1,\epsilon ,\delta)$ , where we added Truncated Laplace noise. If the noised count  $c_{v}$  is too small (at most  $\frac{\Delta}{\epsilon}\cdot \log \left(1 + \frac{e^{\epsilon} - 1}{\delta}\right)$ ), we replace the count  $c_{v}$  with 0.
<sup>1</sup>We remark that the partition is technically not a full partition, because some data points may not be sent to any  $P_{v_i}$ .
Algorithm 2 ANSWER  $(\mathcal{D},\eta_q,q,v)$ : Answering a query
1: Input: data structure  $\mathcal{D} = (\mathcal{T},\{g_v\}_{v\in \mathcal{T}},\{c_v\}_{v\text{leaf}\in \mathcal{T}})$ , parameter  $\eta_{q}$ , query point  $q$ , node  $v$ .  
2: if  $\langle g_v,q\rangle \geq \eta_u$  then  
3: if  $v$  is a leaf node then  
4: Return  $c_v$   
5: else  
6: ans = 0  
7: for  $i = 1$  to  $T$  do  
8:  $v_i\gets$  ith child of  $v$   
9: ans  $\leftarrow$  ans + ANSWER  $(\mathcal{D},\eta_q,q,v_i)$   
10: Return ans
Answering a query: We describe the algorithm in words here, provide pseudocode in Algorithm 2. Given a query  $q$ , we "send"  $q$  to every leaf node  $v = v_{K}$  such that the path  $v_{1}, v_{2}, \ldots, v_{K}$  from the root  $v_{\mathrm{root}}$  satisfies  $\langle g_{v_i}, q \rangle \geq \eta_q$  for all  $1 \leq i \leq K$ . Hence, each query  $q$  will correspond to a subset  $V_{q}$  of leaf nodes. The response to the query  $q$  will be
$$
\sum_ {v \in V _ {q}} c _ {v}.
$$
To improve the accuracy of the algorithm, we repeat this procedure  $O(\log m)$  times if there are  $m$  queries, each initialized to  $(\epsilon', \delta')$ -DP for  $\epsilon' = \epsilon / O(\log m)$  and  $\delta' = \delta / O(\log m)$ . The overall algorithm will still be  $(\epsilon, \delta)$ -DP, by the well-known Basic Composition theorem [DR+14, Corollary 3.15]. For each query, we output the median response of the individual data structure responses. In the analysis, we will ignore this amplification procedure, as by a Chernoff bound it suffices to show that each individual query is answered accurately with at least  $2/3$  probability.
Privacy: The analysis of privacy is quite simple. First, note that the responses to all of the queries do not depend on the data directly, but only depend on the counts  $\{c_v\}$ . If we let  $\hat{c}_v(X) \coloneqq |X \cap P_v|$  and  $\tilde{c}_v(X) \coloneqq |X \cap P_v| + \mathrm{TLap}(1, \epsilon, \delta)$ , then note that  $c_v(X)$  is entirely dependent on  $\tilde{c}_v(X)$ . Hence, it suffices to show the following.
Lemma 3.2 (Privacy). The set  $\{\tilde{c}_v(X)\}_{v\text{leaf}\in \mathcal{T}}$  is  $(\epsilon ,\delta)$ -DP with respect to the data  $X$ . Hence, the entire algorithm is also  $(\epsilon ,\delta)$ -DP, even if we have an arbitrary number of queries.
Proof. Consider two adjacent datasets  $X$ ,  $X'$ , i.e., where we either removed or added a single data point from  $X$  to obtain  $X'$ . Note that the construction partitioning of the dataset into leaves is not dependent on the data, but merely on random vectors  $g_v$  over all nodes  $v$ . Therefore, if we condition on the partitioning of the entire space, the values  $\hat{c}_v(X) = |X \cap P_v|$  and  $\hat{c}_v(X') = |X' \cap P_v|$  are the same for all but at most one leaf  $v$ , which changes by at most 1. The sensitivity of each such  $\hat{c}_v(X)$  is at most 1, which means that we have  $(\epsilon, \delta)$ -privacy loss from  $\tilde{c}_v(X)$ . Because this happens for a single choice of  $v$ , we have  $(\epsilon, \delta)$ -privacy loss in total.
Accuracy: We will only consider the accuracy with respect to a fixed query  $q$  and a single copy of the data structure, and show accuracy holds in expectation. For a set of  $m$  queries, since we repeat this data structure  $O(\log m)$  times and use the median estimate, a Chernoff bound implies all queries will be answered accurately with at least 99% probability.
We prove the following lemma to bound accuracy.
Lemma 3.3 (Accuracy, assuming parameters are set properly). Suppose the following hold, for some fixed choice of parameters  $c$ ,  $\rho$ ,  $K$ ,  $T$ ,  $\eta_q$ ,  $\eta_u$ :
1.  $(T\cdot F(\eta_q))^K\leq n^{\rho +o(1)}$  . (Additive error due to Laplace noise).
2.  $e^{-TF(\eta_u)} = o(1 / K)$ , and  $\frac{G(r,\eta_q,\eta_u)}{F(\eta_u)} = 1 - o(1 / K)$ . (Multiplicative error due to not including points within  $r$  of  $q$ ).
3.  $\left(\frac{G(c r,\eta_{q},\eta_{u})}{F(\eta_{u})}\right)^{K} \leq n^{-1 + \rho + o(1)}$ . (Additive error due to including points not within cr of q).
Then, if ans is the response when querying a fixed  $q$  on a dataset  $X$ , with probability at least  $2/3$ , ans is between
$$
(1 - o (1)) \cdot | X \cap B _ {2} (q, r) | - O \left(\frac {1}{\epsilon} \cdot \log \frac {1}{\delta} \cdot n ^ {\rho + o (1)}\right)
$$
$$
(1 + o (1)) \cdot | X \cap B _ {2} (q, c r) | + O \left(\frac {1}{\epsilon} \cdot \log \frac {1}{\delta} \cdot n ^ {\rho + o (1)}\right).
$$
In other words, we solve the  $(c,r)$ -near neighbor counting problem with probability at least  $2/3$ , up to a multiplicative  $1 \pm o(1)$  factor and an additive  $O\left(\frac{1}{\epsilon} \cdot \log \frac{1}{\delta} \cdot n^{\rho + o(1)}\right)$  factor.
In Subsection 3.4, we will show how to set  $T, K, \eta_q, \eta_u$  so that the three conditions above hold.
Proof. Our error comes from two sources. The first is the Truncated Laplace noise that we add, which we add in  $|V_q|$  locations to compute the answer to query  $q$ . This adds an additive noise of at most  $O\left(\frac{1}{\epsilon} \cdot \log \frac{1}{\delta} \cdot \mathbb{E}|V_q|\right)$  in expectation, as  $|c_v - |X \cap P_v|| \leq O\left(\frac{1}{\epsilon} \cdot \log \frac{1}{\delta}\right)$  with probability 1.
To bound  $\mathbb{E}|V_q|$ , note that it simply equals the sum of the probabilities that  $q \in P_v$  for each leaf  $v$  which is  $T^K$  times the probability that  $q \in P_v$  for a fixed  $v$ . This probability is just  $F(\eta_q)^K$  since each Gaussian along the path to  $v$  is independent. Hence, the first source error has magnitude at most  $O\left(\frac{1}{\epsilon} \cdot \log \frac{1}{\delta} \cdot (T \cdot F(\eta_q))^K\right) \leq O\left(\frac{1}{\epsilon} \cdot \log \frac{1}{\delta} \cdot n^{\rho + o(1)}\right)$  in expectation.
Hence, if we let  $\widehat{ans}$  be the response we would have received if we did not add Laplace noise (i.e., if we used  $\hat{c}_v$  instead of  $c_{v}$ ), then  $\mathbb{E}[|ans - \widehat{ans} |]\leq O\left(\frac{1}{\epsilon}\cdot \log \frac{1}{\delta}\cdot n^{\rho +o(1)}\right)$ , so  $|\text{ans} -\widehat{ans} |\leq O\left(\frac{1}{\epsilon}\cdot \log \frac{1}{\delta}\cdot n^{\rho +o(1)}\right)$  with probability at least 0.9 by Markov's inequality.
The second source of error is that  $\widehat{ans}$  counts the number of points in  $X$  that are in  $P_v$  for some  $v \in V_q$ , whereas we actually want to count the number of points in  $X$  that are within distance  $r$  of  $q$  (tolerating the inclusion of points up to distance  $cr$  of  $q$ ). Hence, we need to make sure of two things, corresponding to not underestimating or overestimating, respectively:
1. Most points in  $X$  within distance  $r$  of  $q$  are mapped to some  $P_v$  where  $v \in V_q$ .
2. Few points in  $X$  that are not within distance  $cr$  of  $q$  are mapped to some  $P_v$  where  $v \in V_q$ .
This way, we will show that with high probability,
$$
(1 - o (1)) \cdot | X \cap B _ {2} (q, r) | \leq \widehat {a n s} \leq | X \cap B _ {2} (q, c r) | + O \left(n ^ {\rho + o (1)}\right),
$$
which is sufficient.
For any point  $p$  of distance  $s$  from  $q$ , we compute the probability that  $p \in P_v$  for some  $v \in V_q$ . Let's start by assuming  $K = 1$ , i.e., we have a depth-1 tree with leaves labeled  $1, 2, \ldots, T$ . In this case there are two possibilities for failure: either  $p$  is not mapped to any  $P_i$  for  $1 \leq i \leq T$ , or  $p$  is mapped to some  $P_i$  but  $q$  is not sent there. Suppose  $p$  is mapped to  $P_1$ : this means  $\langle g_1, p \rangle \geq \eta_u$ . Conditioned on this event, the probability that  $\langle g_1, q \rangle \geq \eta_q$  is precisely  $\mathbb{P}[\langle g_1, q \rangle \geq \eta_q | \langle g_1, p \rangle \geq \eta_u] = \frac{\mathbb{P}[\langle g_1, q \rangle \geq \eta_q, \langle g_1, p \rangle \geq \eta_u]}{\mathbb{P}[\langle g_1, p \rangle \geq \eta_u]} = \frac{G(s, \eta_q, \eta_u)}{F(\eta_u)}$ . Suppose  $p$  is mapped to  $P_i$  for  $i \geq 2$ . Then, the probability that  $\langle g_i, q \rangle \geq \eta_q$  is  $\mathbb{P}[\langle g_i, q \rangle \geq \eta_q | \langle g_i, p \rangle \geq \eta_u, \langle g_j, p \rangle < \eta_u \forall j < i]$ . But even if  $p, q$  are fixed, then the values of  $\langle g_j, p \rangle$  for  $j < i$  are independent of  $(\langle g_i, p \rangle, \langle g_i, q \rangle)$ . Hence, we may remove the conditioning on  $\langle g_j, p \rangle < \eta_u \forall j < i$ , to again say that the conditional probability is  $\frac{G(s, \eta_q, \eta_u)}{F(\eta_u)}$ . Hence, the probability that  $p \in P_v$  for  $v \in V_q$ , conditioned on  $p$  being in some  $P_v$ , is  $\frac{G(s, \eta_q, \eta_u)}{F(\eta_u)}$ . The probability that  $s \notin P_v$  for any  $v$  is  $(1 - F(\eta_u))^T$ , so the overall probability that  $p \in P_v$  for  $v \in V_q$ , for  $K = 1$ , is
$$
\left(1 - \left(1 - F \left(\eta_ {u}\right)\right) ^ {T}\right) \cdot \frac {G \left(s , \eta_ {q} , \eta_ {u}\right)}{F \left(\eta_ {u}\right)}.
$$
Note that  $1 - e^{-TF(\eta_u)} \leq 1 - (1 - F(\eta_u))^T \leq 1$ .
For general depths, note that the Gaussians selected at each level are independent of the previous levels. Hence, the success probability simply raises to the power of  $K$ , or equals
$$
\left(1 - \left(1 - F (\eta_ {u})\right) ^ {T}\right) ^ {K} \cdot \left(\frac {G (s , \eta_ {q} , \eta_ {u})}{F (\eta_ {u})}\right) ^ {K}.
$$
Note that  $1 \geq (1 - e^{-TF(\eta_u)})^K \geq 1 - K \cdot e^{-TF(\eta_u)}$ . Hence, the probability that a point  $p$  of distance  $s$  from  $q$  is mapped to some  $P_v$  where  $v \in V_q$  is between  $(1 - K \cdot e^{-TF(\eta_u)}) \cdot \left( \frac{G(s, \eta_q, \eta_u)}{F(\eta_u)} \right)^K$  and  $\left( \frac{G(s, \eta_q, \eta_u)}{F(\eta_u)} \right)^K$ .
Because  $G(s, \eta_q, \eta_u)$  is an increasing function in  $s$ , the probability at any point of distance  $s \leq r$  from  $q$  is not included in the count  $\widehat{ans}$  is at least  $\left(1 - K \cdot e^{-TF(\eta_u)}\right) \cdot \left(\frac{G(r, \eta_q, \eta_u)}{F(\eta_u)}\right)^K = 1 - o(1)$ , as we are assuming that  $e^{-T \cdot F(\eta_u)} = o(1 / K)$  and  $\frac{G(r, \eta_q, \eta_u)}{F(\eta_u)} = 1 - o(1 / K)$ . Thus, by Markov's inequality, with probability at least 0.9,  $\widehat{ans} \geq (1 - o(1)) \cdot |X \cap B_2(q, r)|$ . Next, the probability that any point of distance  $s \geq cr$  from  $q$  is included in the count  $\widehat{ans}$  is at most  $\left(\frac{G(hr, \eta_q, \eta_u)}{F(\eta_u)}\right)^K$ , which we assume is at most  $n^{-1 + \rho + o(1)}$ . Because there are at most  $n$  points, by Markov's inequality,  $\widehat{ans} \leq |X \cap B_2(q, cr)| + n^{\rho + o(1)}$  with probability at least 0.9.
Runtime: While we have written the algorithm to create a  $T^K$ -sized tree, we can speed up the implementation by only including the necessary parts of the data structure.
We can bound the runtime as follows. Naively, for preprocessing, it takes space and time  $O(T^{K} \cdot d)$  to construct the tree and generate and store the Gaussians. Next, for each point  $p$ , if  $p \in P_v$ , it takes up to  $O(T \cdot d)$  time to determine which child of  $v$  is that  $p$  will be sent to. Since  $p$  is sent to only one node in each partition, this takes total time  $O(K \cdot T \cdot d)$  per point, which means  $O(n \cdot K \cdot T \cdot d)$  total time to partition the points in  $X$ . To improve the preprocessing time, we do not generate the full tree: rather, we only generate a node of the tree (with a corresponding Gaussian  $g_v$ ) if we need to check whether some data point  $p \in X$  is sent to the node, which means we can improve the preprocessing time to  $O(n \cdot K \cdot T \cdot d)$ , without the extra term of  $O(T^{K} \cdot d)$ . Note that if some leaf node  $v$  was not created in our modified implementation, then no point would have been sent to some partition piece  $P_v$ , which means  $|X \cap P_v| + \mathrm{TLap}(1, \frac{\epsilon}{2}, \frac{\delta}{2}) \leq \frac{2}{\epsilon} \cdot \log \left(1 + \frac{e^{\epsilon/2} - 1}{\delta}\right)$ . So, we would have set  $c_v$  to be 0 anyway, which means that this modification does not affect any of the responses to queries.
To answer a query  $q$ , if  $q \in P_v$  (where  $v$  is possibly not a leaf node) it takes  $O(T \cdot d)$  time to determine all of the children of  $v$  that  $q$  will be sent to. In expectation,  $q$  is sent to  $(TF(\eta_q))^{\ell}$  nodes at level  $\ell$ , which means at level  $\ell + 1$  we need to check up to  $(TF(\eta_q))^{\ell} \cdot T$  nodes in expectation. (If a node  $w$  is not created in our implementation, we do not have to check it, since we know  $c_v = 0$  for any leaf  $v$  that is a descendant of  $w$ .) Therefore, the total time it takes to determine all leaf nodes  $q$  is sent to, in expectation, is  $O\left(d \cdot \sum_{\ell=0}^{K-1} (TF(\eta_q))^{\ell} \cdot T\right) = O((TF(\eta_q))^K \cdot K \cdot T \cdot d)$ . Finally, we can add  $c_v$  over all leaf nodes  $v$  that  $q$  is sent to, which takes an additional  $O((TF(\eta_q))^K)$  time in expectation.
Hence, we have the following lemma.
Lemma 3.4 (Runtime). The total preprocessing time of the data structure is  $O(n \cdot K \cdot T \cdot d)$ , and the expected time needed to answer each query is  $O((TF(\eta_q))^K \cdot K \cdot T \cdot d)$ .
3.4 Parameter Settings and Finishing the Accuracy/Runtime Analysis
In this section, we set parameters properly so that Lemmas 3.3 and 3.4 match the goals of Theorem 1.1. We recall that  $c > 1$  is a fixed constant representing the approximation ratio. We assume that  $d = (\log n)^{O(1)}$  and  $r = \Theta \left(\frac{1}{(\log n)^{1/8}}\right)$ . For fixed  $c$  and  $r$  as set above, we set the parameters as follows:
-  $K \coloneqq \sqrt{\ln n}$ .
-  $\eta_u := \sqrt{\frac{\ln n}{K}} \cdot \frac{\lambda}{r}$ , for some constant  $\lambda = \lambda(c)$ , that we will set later.
-  $\eta_q \coloneqq \alpha(r) \cdot \eta_u - 2\beta(r) \cdot \sqrt{\ln K}$ . Recall that  $\alpha(r) = 1 - \frac{r^2}{2}$  and  $\beta(r) = \sqrt{1 - \alpha(r)^2}$ .
$T\coloneqq 10\ln K / F(\eta_u)$
First, we must show that our parameter choices imply that the algorithm is accurate. We prove the following lemma, for which the proof is deferred to Appendix A.
Lemma 3.5 (Accuracy, completed). For an appropriate choice of  $\lambda(c) = \frac{2\sqrt{2}c}{c^2 + 1}$ , all three conditions in Lemma 3.3 hold, with  $\rho = \frac{4c^2}{(c^2 + 1)^2} = O\left(\frac{1}{c^2}\right)$ . Hence, Lemma 3.3 holds without the conditions, for  $\rho = \frac{4c^2}{(c^2 + 1)^2}$ .
Finally, we show that the runtime is good, under our parameter choices.
Lemma 3.6 (Runtime, completed). For the parameters we have defined, assuming  $r = \Theta \left( \frac{1}{(\log n)^{1/8}} \right)$ , the total preprocessing time is  $n^{1 + o(1)} \cdot d$  and the expected time to answer each query is  $n^{\rho + o(1)} \cdot d$ , for  $\rho = \frac{4c^2}{(c^2 + 1)^2}$ .
Proof. The preprocessing time is  $O(n \cdot K \cdot T \cdot d)$ . We know that  $T = \frac{10\ln K}{F(\eta_u)}$ , and that  $\frac{1}{F(\eta_u)} \leq \Omega(\eta_u) \cdot e^{\eta_u^2 / 2} \leq (\log n)^{O(1)} \cdot e^{\ln n \cdot \lambda^2 / (r^2 \cdot K)}$ . By our settings of  $r = \Theta((\log n)^{-1/8})$  and  $K = \sqrt{\ln n}$  and since  $\lambda$  is a constant, this equals  $n^{\Theta(1 / (\log n)^{1/4})} = n^{o(1)}$ . Hence,  $1 / F(\eta_u) = n^{o(1)}$ , and since  $K = O(\sqrt{\log n})$ , the total preprocessing time is  $n^{1 + o(1)} \cdot d$ .
The time to answer each query is  $O((TF(\eta_q))^K \cdot K \cdot T \cdot d)$ . By Lemma 3.5, we know that  $(TF(\eta_q))^K = n^{\rho + o(1)}$ . Moreover, we know that  $K = (\log n)^{O(1)}$ , and we already saw that  $T = 10 \ln K / F(\eta_u) = n^{o(1)}$ . Hence, the time to answer each query is  $n^{\rho + o(1)} \cdot d$ .
Lemmas 3.2, 3.5, and 3.6 will end up being sufficient to prove Theorem 1.1 if we assume the data lies on a  $d = (\log n)^{O(1)}$ -dimensional unit sphere, and if  $r = \Theta((\log n)^{1/8})$ . To finish the analysis, we must show that this assumption can be made without loss of generality, which will follow from an embedding argument of [BRS11]. We defer this argument, and the proof of the remainder of Theorem 1.1, to Appendix A.2.
Remark. We note that the data structure can be implemented so that extra space is only  $O(nd)$ . Instead of generating and storing the Gaussians  $g_v$  independently, we note that in each level  $\ell$ , each group of  $T$  siblings can reuse exactly the same random set of  $T$  Gaussians. The analysis goes through mutatis mutandis as independence between different branches is not needed.
Acknowledgements: PI was supported by the NSF TRIPODS program (award DMS-2022448), Simons Investigator Award, and GIST-MIT Research Collaboration grant.
SN is supported by a NSF Graduate Fellowship (Grant No. 1745302) and a Google PhD Fellowship. AA supported in part by NSF (CCF2008733) and ONR (N00014-22-1-2713).
[Abo18] John M Abowd. The us census bureau adopts differential privacy. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 2867-2867, 2018.
[AI06] Alexandr Andoni and Piotr Indyk. Near-optimal hashing algorithms for approximate nearest neighbor in high dimensions. pages 459-468, 2006.
[ALRW17] Alexandr Andoni, Thijs Laarhoven, Ilya P. Razenshteyn, and Erik Waingarten. Optimal hashing-based time-space trade-offs for approximate near neighbors. In Philip N. Klein, editor, Proceedings of the Twenty-Eighth Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), pages 47-66. SIAM, 2017.
$\left[\mathrm{BEM}^{+}17\right]$  Andrea Bittau, Ülfar Erlingsson, Petros Maniatis, Ilya Mironov, Ananth Raghunathan, David Lie, Mitch Rudominer, Ushasree Kode, Julien Tinnes, and Bernhard Seefeld.
[Sha14] Stephen Shankland. How google tricks itself to protect chrome user privacy. CNET, October, 2014.
[Vad17] Salil Vadhan. The complexity of differential privacy. Tutorials on the Foundations of Cryptography: Dedicated to Oded Goldreich, pages 347-450, 2017.
[WNM23] Tal Wagner, Yonatan Naamad, and Nina Mishra. Fast private kernel density estimation via locality sensitive quantization. 2023.
[XWG10] Xiaokui Xiao, Guozhang Wang, and Johannes Gehrke. Differential privacy via wavelet transforms. IEEE Transactions on knowledge and data engineering, 23(8):1200-1214, 2010.
[ZXX16] Jun Zhang, Xiaokui Xiao, and Xing Xie. Privtree: A differentially private algorithm for hierarchical decompositions. In Proceedings of the 2016 international conference on management of data, pages 155-170, 2016.
A Omitted Proofs from Section 3
A.1 Completion of Accuracy Analysis
Here, we complete the accuracy analysis by proving Theorem 3.5.
Proof of Theorem 3.5. To bound  $(T\cdot F(\eta_q))^K$ , note that  $(10\ln K)^{K}\leq n^{o(1)}$  by our setting of  $K = \sqrt{\ln n}$ . Therefore, it suffices to show that  $(F(\eta_{q}) / F(\eta_{u}))^{K}\leq n^{\rho +o(1)}$ . However, by Proposition 3.1, we can upper bound  $F(\eta_q) / F(\eta_u)\leq O(\eta_u)\cdot e^{(\eta_u^2 -\eta_q^2) / 2}$ . But we know that  $\eta_{u}\leq (\log n)^{O(1)}$  which implies that  $O(\eta_u)^K\leq n^{o(1)}$ . Hence, it suffices to bound  $(e^{(\eta_u^2 -\eta_q^2) / 2})^K$ . Indeed, we can write
$$
\begin{array}{l} \eta_ {u} ^ {2} - \eta_ {q} ^ {2} = \eta_ {u} ^ {2} - (\alpha (r) \cdot \eta_ {u} - 2 \beta (r) \cdot \sqrt {\ln K}) ^ {2} \\ \leq \left(1 - \alpha (r) ^ {2}\right) \cdot \eta_ {u} ^ {2} + 4 \alpha (r) \beta (r) \sqrt {\ln K} \cdot \eta_ {u} \\ \leq \lambda^ {2} \cdot \frac {\ln n}{K} + 4 \lambda \cdot \sqrt {\frac {\ln n \cdot \ln K}{K}} \\ \leq \left(\lambda^ {2} \cdot \frac {\ln n}{K}\right) \cdot (1 + o (1)). \\ \end{array}
$$
Above, the third line follows from the fact that  $1 - \alpha (r)^2\leq r^2$  and  $\alpha (r)\cdot \beta (r)\leq \beta (r)\leq r$ , and the fourth line follows from our definition of  $K$ . Therefore,
$$
\begin{array}{l} \left. \left(T \cdot F \left(\eta_ {q}\right)\right) ^ {K} \leq n ^ {o (1)} \cdot \exp \left(\frac {1}{2} \cdot \left(\lambda^ {2} \cdot \frac {\ln n}{K}\right) \cdot (1 + o (1))\right) ^ {K} \right. \\ = n ^ {o (1)} \cdot \exp \left(\frac {\lambda^ {2}}{2} \cdot \ln n \cdot (1 + o (1))\right) \\ = n ^ {\lambda^ {2} / 2 + o (1)}. \\ \end{array}
$$
Next, note that  $e^{-TF(\eta_u)} = e^{-10\log K} = K^{-10} = o(1 / K)$ , since  $K = \omega(1)$ .
Next, we bound  $\frac{G(r,\eta_q,\eta_u)}{F(\eta_u)}$ . Since  $G(r,\eta_q,\eta_u) = \mathbb{P}[\langle g,q\rangle \geq \eta_q$  and  $\langle g,p\rangle \geq \eta_u]$  for  $\| p - q\| _2 = r$ , we can rewrite this as  $\frac{G(r,\eta_q,\eta_u)}{F(\eta_u)} = \mathbb{P}[\langle g,q\rangle \geq \eta_q|\langle g,p\rangle \geq \eta_u]$  for  $\| p - q\| _2 = r$ . Since  $p,q$  are unit vectors, we can write  $q = \alpha (r)\cdot p + \beta (r)\cdot p^{\prime}$ , where  $p^\prime$  is a unit vector orthogonal to  $p$ . Hence, if we let  $Y = \langle g,p\rangle$  and  $Z = \langle g,p^{\prime}\rangle$ , then  $Y,Z$  are independent standard Gaussians and our goal is to bound  $\mathbb{P}[\alpha (r)\cdot Y + \beta (r)\cdot Z\geq \eta_q|Y\geq \eta_u]$ . But,  $\eta_{q} = \alpha (r)\cdot \eta_{u} - 2\beta (r)\cdot \sqrt{\ln K}$  which means if  $Z\geq -2\sqrt{\ln K}$ , then  $Y\geq \eta_{u}$  automatically implies  $\alpha (r)\cdot Y + \beta (r)\cdot Z\geq \eta_q$ . Therefore, our probability is at least  $\mathbb{P}(Z\geq -2\sqrt{\ln K}) = 1 - \mathbb{P}(Z\geq 2\sqrt{\ln K}) = 1 - o(1 / K)$ , since  $\mathbb{P}(Z\geq 2\sqrt{\ln K}) = O(1 / K^2)$ .
Finally, we must bound  $\left(\frac{G(c r,\eta_{q},\eta_{u})}{F(\eta_{u})}\right)^{K}$ . Note that  $\frac{G(c r,\eta_{q},\eta_{u})}{F(\eta_{u})} = \mathbb{P}[\langle g,q\rangle \geq \eta_{q}|\langle g,p\rangle \geq \eta_{u}]$  for  $\| p - q\| _2 = cr$ , which means  $q = \alpha (cr)\cdot p + \beta (cr)\cdot p'$ . Again, letting  $Y = \langle g,p\rangle$  and  $Z = \langle g,p^{\prime}\rangle$ , we wish to bound
$$
\frac {G \left(c r , \eta_ {q} , \eta_ {u}\right)}{F \left(\eta_ {u}\right)} = \mathbb {P} \left[ \alpha (c r) \cdot Y + \beta (c r) \cdot Z \geq \alpha (r) \cdot \eta_ {u} - 2 \beta (r) \cdot \sqrt {\ln K} | Y \geq \eta_ {u} \right]. \tag {1}
$$
We can rewrite this as
$$
\mathbb {P} \left[ Z \geq \frac {(\alpha (r) - \alpha (c r)) \cdot \eta_ {u} - 2 \beta (r) \cdot \sqrt {\ln K}}{\beta (c r)} - \frac {\alpha (c r) \cdot (Y - \eta_ {u})}{\beta (c r)} \right\rvert   Y \geq \eta_ {u} \Bigg ] \tag {2}
$$
If we write  $W = Y - \eta_{u}$ , note that the PDF of  $W$  is  $\frac{1}{\sqrt{2\pi}} \cdot e^{-(W + \eta_u)^2 / 2}$ . The probability that  $W \geq 0$  is between  $\Omega \left(\frac{1}{\eta_u}\right) \cdot e^{-\eta_u^2 /2}$  and  $e^{-\eta_u^2 /2}$ , which means
$$
\mathbb {P} (W \geq w | W \geq 0) \leq \frac {e ^ {- (\eta_ {u} + w) ^ {2} / 2}}{\Omega (1 / \eta_ {u}) \cdot e ^ {- \eta_ {u} ^ {2} / 2}} \leq O (\eta_ {u}) \cdot e ^ {- w ^ {2} / 2 - w \cdot \eta_ {u}} \leq O (\eta_ {u}) \cdot e ^ {- \eta_ {u} \cdot w}.
$$