SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Alexandr Andoni
Columbia University andoni@cs.columbia.edu
Piotr Indyk
MIT
indyk@mit.edu
Microsoft Research smahabadi@microsoft.com
Shyam Narayanan
MIT shyamsn@mit.edu
Range counting (e.g., counting the number of data points falling into a given query ball) under differential privacy has been studied extensively. However, the current algorithms for this problem are subject to the following dichotomy. One class of algorithms suffers from an additive error that is a fixed polynomial in the number of points. Another class of algorithms allows for polylogarithmic additive error, but the error grows exponentially in the dimension. To achieve the latter, the problem is relaxed to allow a "fuzzy" definition of the range boundary, e.g., a count of the points in a ball of radius  $r$  might also include points in a ball of radius  $cr$  for some  $c > 1$ . In this paper we present an efficient algorithm that offers a sweet spot between these two classes. The algorithm has an additive error that is an arbitrary small power of the data set size, depending on how fuzzy the range boundary is, as well as a small  $(1 + o(1))$  multiplicative error. Crucially, the amount of noise added has no dependence on the dimension. Our algorithm introduces a variant of Locality-Sensitive Hashing, utilizing it in a novel manner.
Differential Privacy (DP) [DMNS06,  $\mathrm{DKM}^{+}06$ , DMNS16] is a widely used tool for preserving the privacy of sensitive personal information. It allows a data structure to provide approximate answers to queries about the data it holds, while ensuring that the removal or addition of a single database entry does not significantly affect the outcome of any analysis. The latter guarantee is accomplished by adding some amount of noise to the answers, so that the data cannot be "reverse-engineered" from the answers to the queries. See Definition 2.1 for the formal setup. The notion has been deployed in many important scenarios in industry [DKY17, EPK14,  $\mathrm{BEM}^{+}17$ , Sha14,  $\mathrm{G}^{+}16$ ] as well as the U.S. census [Abo18].
One of the key data analysis problems studied under differential privacy is range counting. Here, the goal is to construct a data structure that answers queries about the total number of data items in a database satisfying a given property. Formally, given a multiset  $X$  of  $n$  elements from a universe  $\mathcal{U}$ , the goal of range counting queries is to report the number of points within a given range  $Q \in \mathcal{Q}$  from a prespecified class of query ranges  $\mathcal{Q}$ . For example, this could correspond to retrieving the number of users in a particular geographic location, or a number of patients with given symptoms.
Differentially private range counting has been studied extensively, for many classes of queries. For example, the case of  $\mathcal{Q} = \mathcal{U}$ , i.e., when the queries ask for the number of occurrences of a given element in the database, this corresponds to the well-known histogram problem [HKR12, EPK14].
Figure 1: An illustration of approximate near neighbor query, with parameters  $c$  and  $r$ .
More generally, a large body of research has focused on the case where universe  $\mathcal{U}$  is the  $d$ -dimensional space  $\mathbb{R}^d$  (or its discretization), and  $\mathcal{Q}$  consists of natural geometric objects, like rectangles, half-planes or balls. The seminal work of [MN12] demonstrated a strong relationship between the amount of additive noise necessary to guarantee the privacy, and the discrepancy of the space of queries  $\mathcal{Q}$ . Unfortunately, except for a few classes of queries such as axis-parallel rectangles, most natural geometric objects have discrepancy that is polynomial in the number of data points  $n$ . This means that, in order to guarantee privacy, it is necessary to distort the counts by an additive term  $\Theta(n^\rho)$ , for a fixed constant  $\rho > 0$  that depends on  $\mathcal{Q}$ .
To alleviate this issue, [HY21] introduced differentially private data structures for approximate range queries. For simplicity, we recap their notion for the special case of ball queries that we consider in the rest of the paper. We assume that  $\mathcal{U}$  is a metric space equipped with a distance function dist. For a radius parameter  $r$ , we define the query family  $\mathcal{Q} = \{B(q,r)\colon q\in \mathcal{U}\}$ , where  $B(q,r) = \{p\in \mathcal{U}\colon \mathrm{dist}(p,q)\leq r\}$  is the ball of radius  $r$  around  $q$ . In other words, in the exact problem, the goal is to count the number of data points that are "near" the given query point. In turn, the approximate problem is parameterized by an approximation factor  $c\geq 1$ , and the algorithm is allowed to report any number between  $|X\cap B(q,r)|$  and  $|X\cap B(q,cr)|$ . Figure 1 illustrates this notion. We will refer to the approximate version as the  $(c,r)$ -near neighbor counting problem.
The main result of [HY21] shows that, if one allows approximate queries as defined above, then one can construct a data structure that guarantees privacy while adding much less noise. Specifically, if  $\mathcal{U} = [u]^d$  (i.e., the universe is the discrete  $d$ -dimensional space) for some fixed  $d = O(1)$  and  $\mathrm{dist}(p,q) = \| p - q\|_s$  for some  $\ell_s$  norm, then their algorithm achieves privacy while adding noise of magnitude roughly  $(O(1 / \alpha))^d\cdot \mathrm{polylog}(u + n)$ , where  $0 < \alpha < 1$  is a constant that depends on  $c > 1$ . This guarantees that the noise amount is only polylogarithmic, as long as the dimension  $d$  is constant. However, the exponential dependence of the noise bound on the dimension precludes the applicability of this algorithm to the common case of high-dimensional data sets.
Our contribution Our main result is an efficient data structure that can approximate the number of near neighbors to any query point over Euclidean  $(\ell_2)$  space, with differential privacy (see Preliminaries for the formal of definition of privacy). Specifically, for any fixed radius  $r$ , and any series of query points  $q$ , the data structure can privately answer  $(c,r)$ -near neighbor counting queries, where we are allowed a small additive error  $(n^{\rho}$  for a small constant  $\rho$ ) and multiplicative error  $1 \pm o(1)$ . Let  $B_p(q,r)$  denote the  $\ell_p$  ball of radius  $r$  around  $q$ .
Theorem 1.1. Fix a radius  $r > 0$  and constant  $c \geq 1$ , and privacy parameters  $1 > \epsilon, \delta > 0$ . For any integer  $n$  and dimension  $d$ , there exists a differentially private algorithm that, on a dataset  $X \subset \mathbb{R}^d$  of at most  $n$  points, creates a data structure that can answer  $m$  (non-adaptive) queries and that satisfies the following properties.
1. (Privacy) The full output of the algorithm is  $(\epsilon, \delta)$ -differentially private.
2. (Accuracy) With high probability, for every non-adaptive query  $q$ , the output  $ans(q)$  is between
$$
(1 - o (1)) \cdot | X \cap B _ {2} (q, r) | - n ^ {\rho + o (1)} \cdot \frac {\log (\log m / \delta) \cdot \log m}{\epsilon}
$$
$$
(1 + o (1)) \cdot | X \cap B _ {2} (q, c r) | + n ^ {\rho + o (1)} \cdot \frac {\log (\log m / \delta) \cdot \log m}{\epsilon},
$$
$$
w h e r e \rho = \frac {4 c ^ {2}}{(c ^ {2} + 1) ^ {2}} = O (\frac {1}{c ^ {2}}).
$$
3. (Runtime) The preprocessing time is  $O(n^{1 + o(1)} \cdot d \cdot \log m)$ , and the expected runtime for answering each query is  $O(n^{\rho + o(1)} \cdot d \cdot \log m)$ .
The key two features of the above result are as follows. First, the amount of additive noise is controlled by the amount of allowed approximation: the exponent  $\rho$  tends to 0 as  $c$  grows large. This makes it possible to trade the "proximity" error  $c$  and the "privacy" error  $n^{\rho}$  to achieve the best results. Second, the magnitude of added noise does not depend on the dimension; in particular, it avoids the exponential dependence of [HY21].
Finally, we remark that our algorithm can be extended to alternative/more general settings:
1.  $\ell_1$  metric: If we considered the balls  $B_{1}(q,r)$  rather than  $B_{2}(q,r)$ , we can still obtain a similar result, at the cost of having  $\rho \approx \frac{4c}{(c + 1)^2} = O\left(\frac{1}{c}\right)$ . This is based on the fact that the  $\ell_1$  metric embeds near-isometrically into the squared  $\ell_2$  metric.
2. Pure differential privacy: We can modify our algorithm to handle pure-DP queries (i.e., when  $\epsilon = 0$ ), while still having good accuracy. In this case, however, the algorithm, while fast in expectation, has some small probability of being very slow.
3. Large  $\epsilon$ : Our results also hold even when  $1 \leq \epsilon \leq n^{o(1)}$ .
4. Adaptive queries: Our method also extends to adaptive queries, at the expense of the runtime and additive error multiplying by a factor of  $\mathrm{poly}(d\log R / r)$ , where  $R$  is some promised bound on the radius of the dataset.
5.  $k$ -NN queries: Finally, we note that using similar approach to that of [HY21], one can get a data structure for finding an approximation to the distance of the query to its  $k$ -nearest neighbor. However, as opposed to [HY21], our data structure works only for a fixed value of  $r$ , and thus one needs to build separate data structures for various choices of  $r$  leading to worse privacy guarantees which depend on the aspect ratio.
We discuss extensions 1-4 more formally in Appendix C.
A natural question is whether the tradeoff between approximation ratio  $c$  and additive error  $n^{\rho}$  is necessary, or whether one can obtain  $c = O(1)$  and  $\rho$  to be an arbitrarily small constant. While we are unable to prove such a lower bound in the Euclidean setting, we show that under the closely related  $\ell_{\infty}$  norm for approximation ratio  $c = 3 - o(1)$ , one must have an additive error of  $n^{\Omega(1)}$ .
Theorem 1.2. For sufficiently large  $n$ , there exists  $d = C\log n$  for a large constant  $C$ , fixed constants  $1 > \epsilon, \delta, \rho > 0$ , and  $n$  query points  $Q = \{q_1, \ldots, q_n\}$ , with the following property. For any (arbitrarily small)  $\alpha > 0$  and any differentially private algorithm  $\mathcal{A}$  that acts on a dataset  $X \subset \mathbb{R}^d$  of at most  $n$  points, and outputs  $\{ans(q)\}_{q \in Q}$ , must have
$$
\mathbb {P} \big (\forall q \in Q, | B _ {\infty} (q, 0. 5) | - n ^ {\rho} \leq a n s (q) \leq | B _ {\infty} (q, 1. 5 - \alpha) | + n ^ {\rho} \big) <   \frac {2}{3}.
$$
Theorem 1.2 provides some evidence that a tradeoff may be necessary even in the  $\ell_2$  case: proving such a result is an interesting open problem. We defer the proof of Theorem 1.2 to Appendix B.
Overview of the techniques. Our high-level approach is similar to that of the low-dimensional approximate range query data structure of [HY21], and consists of the following steps. First, we design a partition of  $\mathbb{R}^d$ , and count the number of points that fall into each cell in the partition. Then we add Laplace noise to each cell to ensure the privacy of the counts. Finally, for each query  $q$ , we approximate the ball  $B(q,r)$  using the cells in the partition, and report the sum of the (noisy) counts of the approximating cells.
However, applying this approach as described requires using an exponential (in  $d$ ) number of partition cells to approximate a query ball  $B(q,r)$ , leading to the exponential error bound. To overcome this issue, we resort to randomized space partitions obtained using Locality-Sensitive Hashing (LSH) techniques [IM98]. These partitions have the property that, for any two points  $p$  and  $q$ , the probability that both of them belong in the same partition cell depends on the distance  $\| p - q\|_2$ . As shown in [Pan06] in the context of (non-private) approximate near neighbor search, enumerating a bounded
number  $(n^{\rho})$  of cells in such partition makes it possible to report each data point from the range  $B(q,r)$  with probability at least  $1 / \mathrm{poly}(\log n)$ . To amplify the probability of correctness, the process is repeated  $O(\mathrm{poly}\log n)$  times, using a different random partition in each repetition.
Unfortunately, the aforementioned approach does not appear to yield an efficient algorithm for approximately counting the number of points in  $B(q,r)$ . The main issue is that, as indicated above, when enumerating the  $n^{\rho}$  cells corresponding to a query's ball, we are guaranteed to include only a  $1 / \mathrm{poly}(\log n)$  fraction of points, leading to a large approximation factor. The standard way to deal with this — using multiple partitions — yields inconsistent counts which are difficult to aggregate. A number of follow-up works showed improved bounds on exponent  $\rho$ , but all of them similarly have a guarantee of identifying only  $\mathrm{a} \leq 1 / \mathrm{poly}(\log n)$  fraction of points [LJW+07, AI06, Kap15, Chr17, ALRW17]. To overcome these obstacles, we show how we can adapt the algorithm from [ALRW17], which yield an approximate near neighbor data structure with the best known tradeoff between space and time. However, their randomized algorithm does not yield partitions (technically, they construct so-called locality-sensitive filters, which are not partitions).
Our main technical contribution is two-fold. First, we show it is possible to "force" the algorithm of [ALRW17] to construct proper partitions. The quality of those partitions is comparable to the original structures, at the price of a more involved analysis. We then use those partitions to compute the counts and approximate the query ball as outlined at the beginning of this section. Second, we show that it is also possible to modify the algorithm so that the probability of including each qualifying point into the count is at least  $1 - o(1)$ . This is of separate interest, as this yields the most efficient algorithm for approximate nearest neighbor search with space  $O(nd)$ , improving over [Kap15].
The problem of range counting when the query class  $\mathcal{Q}$  is arbitrary, can be solved with an optimal error bound of  $\sqrt{n} \cdot \mathrm{poly}(1 / \epsilon, \log \frac{|\mathcal{Q}| \cdot |\mathcal{U}|}{\delta})$  [HR10]. In the setting of approximate near-neighbor queries, the size of  $\mathcal{Q}$  and  $\mathcal{U}$  can be thought of as at most exponential in the dimension  $d$ , by using a standard  $\epsilon$ -net technique. Hence, their error is in fact smaller than ours when  $\frac{4c^2}{(c^2 + 1)^2} \geq \frac{1}{2}$ , or equivalently, when  $c \leq 1 + \sqrt{2}$ . However, whenever an approximation of  $c > 1 + \sqrt{2}$  is acceptable, our error is smaller than [HR10]. Moreover, our approach is computationally efficient, whereas their method has runtime linear in the size of  $\mathcal{U}$ , which is very inefficient.
When the query class is restricted to points in  $\mathcal{U}$ , i.e., histogram queries, then the best achievable error is  $O\left(\frac{1}{\epsilon} \cdot \min \{\log |\mathcal{U}|, \log (1 / \delta)\}\right)$  [Vad17]. When the points are in 1-dimensional Euclidean space taking values in  $[u]$ , and  $\mathcal{Q}$  is a set of intervals, [DNPR10] got an  $(\epsilon, 0)$ -DP algorithm with error  $O\left(\frac{1}{\epsilon} \cdot \log^{1.5} u\right)$  and a lower bound of  $\Omega \left(\frac{\log u}{\epsilon}\right)$ . When resorting to  $(\epsilon, \delta)$ -DP, [BNSV15] showed an algorithm with error  $2^{(1 + o(1)) \log^* u} \log (1 / \delta) / \epsilon$  and a lower bound of  $\Omega (\log^* u \cdot \log (1 / \delta) / \epsilon)$  for  $e^{-\epsilon n / \log^* n} \leq \delta \leq 1 / n^2$ . The problem for the axis-parallel rectangles have been further studied [CSS11, DNRR15] where polylogarithmic error bounds were obtained.
More generally, [MN12, NTZ13] showed an equivalence between the error and the discrepancy of the range space  $\mathcal{Q}$ . Given that the discrepancy of many natural range spaces such as half-spaces, simplices, and spheres is  $n^{\alpha(d)}$  where  $\alpha(d) \in [1/4, 1/2]$  is a constant depending on  $d$  [Mat99], it rules out such a polylogarithmic error for these range spaces. For non-convex ranges, the discrepancy already becomes  $\sqrt{n}$  reaching the threshold for arbitrary query ranges.
There are a number of papers based on space decompositions that provide a DP data structure for range counting that perform relatively well in practice  $\left[\mathrm{CPS}^{+}12\right.$ , HRMS09, LHMW14, QYL13a, QYL13b, XWG10, XXX16] but perform poorly on high-discrepancy point sets.
Finally, we note that several prior works used LSH for differentially private kernel estimation [CS20, WNM23]. However, those works do not seem to be applicable to ball range queries, which correspond to uniform kernels.
Range Counting: In the range counting problems, given a point set  $X \subseteq \mathcal{U}$ , where  $\mathcal{U}$  is a universe, the goal is to construct a differentially private data structure that for any query  $Q \subseteq \mathcal{U}$  from a certain query family  $\mathcal{Q}$ , one can compute  $|Q \cap X|$ .
Here we focus on Near Neighbor counting queries in high dimensions. In particular  $\mathcal{U}$  is a metric space equipped with a distance function dist. Given a prespecified radius parameter  $r$ , the query family  $\mathcal{Q} = \{B(q,r): q \in \mathcal{U}\}$ , where  $B(q,r) = \{p \in \mathcal{U}: \mathrm{dist}(p,q) \leq r\}$  is the ball of radius  $r$  around  $q$ . In other words, in this problem the goal is to count the number of neighbors of a given query point.
Similar to the work of [HY21], we consider the approximate variant of the problem where the points within distance  $r$  and  $cr$  of the query can be either counted or discarded. In particular for an approximation factor  $c \geq 1$ , any number between  $|X \cap B(q, r)|$  and  $|X \cap B(q, cr)|$  is valid.
Differential Privacy (DP): For two datasets  $X$  and  $X'$ , we use  $X \sim X'$  to mean that they are neighboring datasets, i.e., one of them can be obtained from the other by an addition of a single element (in our problem a point).
Definition 2.1. (Differential Privacy  $[DR^{+}14]$ ). For  $\epsilon >0$ ,  $\delta \geq 0$ , a randomized algorithm  $\mathcal{A}$  is  $(\epsilon, \delta)$ -differentially private  $((\epsilon, \delta)$ -DP) if for any two datasets  $X \sim X'$ , and any possible outcome of the algorithm  $S \subseteq \text{Range}(\mathcal{A})$ ,  $\mathbb{P}[A(X) \in S] \leq e^{\epsilon} \cdot \mathbb{P}[A(X') \in S] + \delta$ . When  $\delta = 0$ , the algorithm is said to have pure differential privacy.
The sensitivity of a function  $f$  is defined to be  $\Delta_f = \max_{X \sim X'} |f(X) - f(X')|$ .
We use  $\mathrm{Lap}(\lambda)$  to denote the Laplace distribution with parameter  $\lambda$  with PDF  $\mathbb{P}[Z = z] = \frac{1}{2\lambda} e^{-|z| / \lambda}$ , which has mean 0 and variance  $\lambda^2$ . We also use  $\mathrm{TLat}(\Delta, \epsilon, \delta)$  to denote the Truncated Laplace distribution with PDF proportional to  $e^{-|z|\cdot \epsilon /\Delta}$  on the region  $[-B,B]$ , where  $B = \frac{\Delta}{\epsilon}\cdot \log \left(1 + \frac{e^{\epsilon} - 1}{2\delta}\right)$ .
Lemma 2.2 ((Truncated) Laplace Mechanism  $[\mathrm{DR}^{+}14$ , GDGK20]). Given a numeric function  $f$  that takes a dataset  $X$  as the input, and has sensitivity  $\Delta$ , the mechanism that outputs  $f(X) + Z$  where  $Z \sim \operatorname{Lap}(\Delta/\epsilon)$  is  $(\epsilon, 0)$ -DP. In addition, if  $\epsilon, \delta \leq \frac{1}{2}$ ,  $f(X) + Z$ , where  $Z \sim \mathrm{TLap}(\Delta, \epsilon, \delta)$ , is  $(\epsilon, \delta)$ -DP. Moreover, the Truncated Laplace mechanism is always accurate up to error  $B$ .
3 Upper Bound
In this section, we describe the algorithm and perform the majority of the analysis for Theorem 1.1. We defer certain parts to Appendix A, and finish the proof there.
The algorithm we develop is inspired by the data-independent approximate nearest neighbor algorithm in [ALRW17]. We will use similar notation to their paper as well.
Define  $S^{d - 1}$  to be the  $(d - 1)$ -dimensional unit sphere in  $\mathbb{R}^d$ . For any parameter  $r \in (0,2)$ , let  $\alpha(r) = 1 - \frac{r^2}{2}$  and  $\beta(r) = \sqrt{1 - \alpha(r)^2}$  be the cosine and the sine, respectively, of the angle between two points in  $S^{d - 1}$  of distance  $r$ . For any parameter  $\eta > 0$ , define
$$
F(\eta):= \underset {g\sim \mathcal{N}(0,I)}{\mathbb{P}}[\langle g,u\rangle \geq \eta ],
$$
where  $u \in S^{d-1}$  is an arbitrary point on the unit sphere. Note that  $F(\eta)$  is independent of  $u$  by the spherical symmetry of Gaussians, and in fact equals the probability that a univariate standard Normal exceeds  $\eta$ . Next, for any parameters  $r \in (0, 2)$  and  $\eta_q, \eta_u \in \mathbb{R}$ , we define
$$
G(r,\eta_{q},\eta_{u}):= \underset {g\sim \mathcal{N}(0,I)}{\mathbb{P}}[\langle g,q\rangle \geq \eta_{q}\text{and}\langle g,p\rangle \geq \eta_{u}],
$$
where  $p, q \in S^{d-1}$  are points that have distance exactly  $r$  between them. As with  $F$ ,  $G$  has no dependence on the specific points  $p, q$  but only on the distance  $r = \|p - q\|_2$  between them. Note also that  $G(r, \eta_q, \eta_u)$  is non-increasing in  $r$ , even if  $\eta_q$  or  $\eta_u$  is negative.
Next, we note the following well-known bounds for  $F$ .
Algorithm 1 Data Structure
1: Input: data  $X$  of size at most  $n$ , privacy parameters  $\epsilon, \delta$ , parameters  $T, K, \eta_u$ .  
2: Output: data structure  $\mathcal{D} = (\mathcal{T}, \{g_v\}_{v \in \mathcal{T}}, \{c_v\}_{v \text{leaf} \in \mathcal{T}})$ .  
3: Create  $\mathcal{T}$ :  $T$ -ary tree of depth  $K$ , with root  $v_{\text{root}}$ .  
4: for each node  $v \in \mathcal{T}$  do  
5:  $g_v \gets \mathcal{N}(0, I_d)$ . {Random  $d$ -dimensional Gaussian}  
6:  $c_v \gets \emptyset$ . {Counts number of points assigned to each node}  
7: for point  $p \in X$  do  
8:  $v \gets v_{\text{root}}$ .  
9: for  $\ell = 0$  to  $K - 1$  do  
10: if  $\text{depth}(v) < \ell$  then  
11: BREAK {Break out of "for  $\ell = 0$  to  $K - 1$  loop (  $p$  failed to map to a child)}  
12: for  $i = 1$  to  $T$  do  
13:  $v_i \gets i$ th child of  $v$   
14: if  $\langle g_{v_i}, p \rangle \geq \eta_u$  then  
15:  $c_{v_i} \gets c_{v_i} + 1$   
16:  $v \gets v_i$   
17: BREAK {Break out of "for  $i = 1$  to  $T$  loop}  
18: for each leaf node  $v \in \mathcal{T}$  do  
19:  $c_v \gets c_v + \text{TLap}(1, \frac{\epsilon}{2}, \frac{\delta}{2})$ .  
20: if  $c_v \leq \frac{2}{\epsilon} \cdot \log(1 + \frac{e^{\epsilon/2} - 1}{\delta})$  then  
21:  $c_v \gets 0$
Proposition 3.1. For  $\eta \geq 1$
$$
\Omega \left(\frac {1}{\eta}\right) \cdot e ^ {- \eta^ {2} / 2} \leq F (\eta) \leq e ^ {- \eta^ {2} / 2}.
$$
In addition, as  $\eta \to \infty$ ,  $F(\eta) \leq o(1) \cdot e^{-\eta^2 / 2}$ .
3.2 Algorithm for data on the sphere
We now present an algorithm that obtains the desired privacy and accuracy guarantees, assuming both the data points and query points are on a unit sphere and the radius  $r$  is not too small. The initial algorithm description will not necessarily run in polynomial time, but we will later describe a simple modification to make this algorithm efficient. Finally, in Appendix A.2 we will modify the algorithm to work for general  $r$  and in Euclidean space, rather than just on the unit sphere. This follows from known embedding methods [BRS11], which is why we defer it to the Appendix.
Data Structure: We describe the data structure in words here, provide pseudocode in Algorithm 1. Fix  $K$  and  $T$  to be positive integers, to be set later. We generate a data structure, which is a  $T$ -ary tree of depth  $K$ . We index the levels as  $0, 1, \dots, K$ , where level 0 is the root and level  $K$  consists of the  $T^K$  leaves. For each node  $v$  except the root, we generate an independent  $d$ -dimensional standard Gaussian  $g_v \sim \mathcal{N}(0, I)$ . In addition, we will define two parameters  $\eta_q$  and  $\eta_u$ , to be set later.