Title: Differentially Private Approximate Near Neighbor Counting in High Dimensions
Venue: NeurIPS 2023
Accepted: yes
Keywords: Differential Privacy

Near Neighbor Search

Locality Sensitive Hashing

Data Structures

Range Query
Abstract: Range counting (e.g., counting the number of data points falling into a given query ball) under differential privacy has been studied extensively. However, the current algorithms for this problem are subject to the following dichotomy. One class of algorithms suffers from an additive error that is a fixed polynomial in the number of points. Another class of algorithms allows for polylogarithmic additive error, but the error grows exponentially in the dimension. To achieve the latter, the problem is relaxed to allow a “fuzzy” definition of the range boundary, e.g., a count of the points in a ball of radius $r$ might also include points in a ball of radius $cr$ for some $c>1$. In this paper we present an efficient algorithm that offers a sweet spot between these two classes. The algorithm has an additive error that is an arbitrary small power of the data set size, depending on how fuzzy the range boundary is, as well as a small ($1+o(1)$) multiplicative error. Crucially, the amount of noise added has no dependence on the dimension. Our algorithm introduces a variant of Locality-Sensitive Hashing, utilizing it in a novel manner.

Differentially Private Approximate Near Neighbor Counting in High Dimensions

Abstract

Range counting (e.g., counting the number of data points falling into a given query ball) under differential privacy has been studied extensively. However, the current algorithms for this problem are subject to the following dichotomy. One class of algorithms suffers from an additive error that is a fixed polynomial in the number of points. Another class of algorithms allows for polylogarithmic additive error, but the error grows exponentially in the dimension. To achieve the latter, the problem is relaxed to allow a 'fuzzy' definition of the range boundary, e.g., a count of the points in a ball of radius r might also include points in a ball of radius cr for some c > 1 . In this paper we present an efficient algorithm that offers a sweet spot between these two classes. The algorithm has an additive error that is an arbitrary small power of the data set size, depending on how fuzzy the range boundary is, as well as a small ( 1 + o (1) ) multiplicative error. Crucially, the amount of noise added has no dependence on the dimension. Our algorithm introduces a variant of Locality-Sensitive Hashing, utilizing it in a novel manner.

1 Introduction

Differential Privacy (DP) [DMNS06, DKM + 06, DMNS16] is a widely used tool for preserving the privacy of sensitive personal information. It allows a data structure to provide approximate answers to queries about the data it holds, while ensuring that the removal or addition of a single database entry does not significantly affect the outcome of any analysis. The latter guarantee is accomplished by adding some amount of noise to the answers, so that the data cannot be "reverse-engineered" from the answers to the queries. See Definition 2.1 for the formal setup. The notion has been deployed in many important scenarios in industry [DKY17, EPK14, BEM + 17, Sha14, G + 16] as well as the U.S. census [Abo18].

One of the key data analysis problems studied under differential privacy is range counting . Here, the goal is to construct a data structure that answers queries about the total number of data items in a database satisfying a given property. Formally, given a multiset X of n elements from a universe U , the goal of range counting queries is to report the number of points within a given range Q ∈ Q from a prespecified class of query ranges Q . For example, this could correspond to retrieving the number of users in a particular geographic location, or a number of patients with given symptoms.

Differentially private range counting has been studied extensively, for many classes of queries. For example, the case of Q = U , i.e., when the queries ask for the number of occurrences of a given element in the database, this correspond to the well-known histogram problem [HKR12, EPK14].

◦

q

Must be included

May be included

Cannot be included

•

•

•

•

•

•

•

•

•

•

•

•

•

r

cr

More generally, a large body of research has focused on the case where universe U is the d -dimensional space R d (or its discretization), and Q consists of natural geometric objects, like rectangles, halfplanes or balls. The seminal work of [MN12] demonstrated a strong relationship between the amount of additive noise necessary to guarantee the privacy, and the discrepancy of the space of queries Q . Unfortunately, except for a few classes of queries such as axis-parallel rectangles, most natural geometric objects have discrepancy that is polynomial in the number of data points n . This means that, in order to guarantee privacy, it is necessary to distort the counts by an additive term Θ( n ρ ) , for a fixed constant ρ > 0 that depends on Q .

To alleviate this issue, [HY21] introduced differentially private data structures for approximate range queries. For simplicity, we recap their notion for the special case of ball queries that we consider in the rest of the paper. We assume that U is a metric space equipped with a distance function dist . For a radius parameter r , we define the query family Q = { B ( q, r ): q ∈ U} , where B ( q, r ) = { p ∈ U : dist( p, q ) ≤ r } is the ball of radius r around q . In other words, in the exact problem, the goal is to count the number of data points that are 'near' the given query point. In turn, the approximate problem is parameterized by an approximation factor c ≥ 1 , and the algorithm is allowed to report any number between | X ∩ B ( q, r ) | and | X ∩ B ( q, cr ) | . Figure 1 illustrates this notion. We will refer to the approximate version as the ( c, r ) -near neighbor counting problem.

The main result of [HY21] shows that, if one allows approximate queries as defined above, then one can construct a data structure that guarantees privacy while adding much less noise. Specifically, if U = [ u ] d (i.e., the universe is the discrete d -dimensional space) for some fixed d = O (1) and dist( p, q ) = ∥ p -q ∥ s for some ℓ s norm, then their algorithm achieves privacy while adding noise of magnitude roughly ( O (1 /α )) d · polylog ( u + n ) , where 0 < α < 1 is a constant that depends on c > 1 . This guarantees that the noise amount is only polylogarithmic, as long as the dimension d is constant. However, the exponential dependence of the noise bound on the dimension precludes the applicability of this algorithm to the common case of high-dimensional data sets.

Our contribution Our main result is an efficient data structure that can approximate the number of near neighbors to any query point over Euclidean ( ℓ 2 ) space, with differential privacy (see Preliminaries for the formal of definition of privacy). Specifically, for any fixed radius r , and any series of query points q , the data structure can privately answer ( c, r ) -near neighbor counting queries, where we are allowed a small additive error ( n ρ for a small constant ρ ) and multiplicative error 1 ± o (1) ). Let B p ( q, r ) denote the ℓ p ball of radius r around q .

Theorem 1.1. Fix a radius r > 0 and constant c ≥ 1 , and privacy parameters 1 > ϵ, δ > 0 . For any integer n and dimension d , there exists a differentially private algorithm that, on a dataset X ⊂ R d of at most n points, creates a data structure that can answer m (non-adaptive) queries and that satisfies the following properties.

(Privacy) The full output of the algorithm is ( ϵ, δ ) -differentially private.

(Accuracy) With high probability, for every non-adaptive query q , the output ans ( q ) is between

(Runtime) The preprocessing time is O ( n 1+ o (1) · d · log m ) , and the expected runtime for answering each query is O ( n ρ + o (1) · d · log m ) .

The key two features of the above result are as follows. First, the amount of additive noise is controlled by the amount of allowed approximation: the exponent ρ tends to 0 as c grows large. This makes it possible to trade the 'proximity' error c and the 'privacy' error n ρ to achieve the best results. Second, the magnitude of added noise does not depend on the dimension; in particular, it avoids the exponential dependence of [HY21].

Finally, we remark that our algorithm can be extended to alternative/more general settings:

ℓ 1 metric: If we considered the balls B 1 ( q, r ) rather than B 2 ( q, r ) , we can still obtain a similar result, at the cost of having ρ ≈ 4 c ( c +1) 2 = O ( 1 c ) . This is based on the fact that the ℓ 1 metric embeds near-isometrically into the squared ℓ 2 metric.

Pure differential privacy: We can modify our algorithm to handle pure-DP queries (i.e., when ϵ = 0 ), while still having good accuracy. In this case, however, the algorithm, while fast in expectation, has some small probability of being very slow.

Large ϵ : Our results also hold even when 1 ≤ ϵ ≤ n o (1) .

Adaptive queries: Our method also extends to adaptive queries, at the expense of the runtime and additive error multiplying by a factor of poly( d log R/r ) , where R is some promised bound on the radius of the dataset.

k -NN queries: Finally, we note that using similar approach to that of [HY21], one can get a data structure for finding an approximation to the distance of the query to its k -nearest neighbor. However, as opposed to [HY21], our data structure works only for a fixed value of r , and thus one needs to build separate data structures for various choices of r leading to worse privacy guarantees which depend on the aspect ratio.

We discuss extensions 1-4 more formally in Appendix C.

A natural question is whether the tradeoff between approximation ratio c and additive error n ρ is necessary, or whether one can obtain c = O (1) and ρ to be an arbitrarily small constant. While we are unable to prove such a lower bound in the Euclidean setting, we show that under the closely related ℓ ∞ norm for approximation ratio c = 3 -o (1) , one must have an additive error of n Ω(1) .

Theorem 1.2. For sufficiently large n , there exists d = C log n for a large constant C , fixed constants 1 > ϵ, δ, ρ > 0 , and n query points Q = { q 1 , . . . , q n } , with the following property. For any (arbitrarily small) α > 0 and any differentially private algorithm A that acts on a dataset X ⊂ R d of at most n points, and outputs { ans ( q ) } q ∈ Q , must have

Theorem 1.2 provides some evidence that a tradeoff may be necessary even in the ℓ 2 case: proving such a result is an interesting open problem. We defer the proof of Theorem 1.2 to Appendix B.

Overview of the techniques. Our high-level approach is similar to that of the low-dimensional approximate range query data structure of [HY21], and consists of the following steps. First, we design a partition of R d , and count the number of points that fall into each cell in the partition. Then we add Laplace noise to each cell to ensure the privacy of the counts. Finally, for each query q , we approximate the ball B ( q, r ) using the cells in the partition, and report the sum of the (noisy) counts of the approximating cells.

However, applying this approach as described requires using an exponential (in d ) number of partition cells to approximate a query ball B ( q, r ) , leading to the exponential error bound. To overcome this issue, we resort to randomized space partitions obtained using Locality-Sensitive Hashing (LSH) techniques [IM98]. These partitions have the property that, for any two points p and q , the probability that both of them belong in the same partition cell depends on the distance ∥ p -q ∥ 2 . As shown in [Pan06] in the context of (non-private) approximate near neighbor search, enumerating a bounded

number ( n ρ ) of cells in such partition makes it possible to report each data point from the range B ( q, r ) with probability at least 1 / poly(log n ) . To amplify the probability of correctness, the process is repeated O (poly log n ) times, using a different random partition in each repetition.

Unfortunately, the aforementioned approach does not appear to yield an efficient algorithm for approximately counting the number of points in B ( q, r ) . The main issue is that, as indicated above, when enumerating the n ρ cells corresponding to a query's ball, we are guaranteed to include only a 1 / poly(log n ) fraction of points, leading to a large approximation factor. The standard way to deal with this - using multiple partitions - yields inconsistent counts which are difficult to aggregate. A number of follow-up works showed improved bounds on exponent ρ , but all of them similarly have a guarantee of identifying only a ≤ 1 / poly(log n ) fraction of points [LJW + 07, AI06, Kap15, Chr17, ALRW17]. To overcome these obstacles, we show how we can adapt the algorithm from [ALRW17], which yield an approximate near neighbor data structure with the best known tradeoff between space and time. However, their randomized algorithm does not yield partitions (technically, they construct so-called locality-sensitive filters , which are not partitions).

Our main technical contribution is two-fold. First, we show it is possible to 'force' the algorithm of [ALRW17] to construct proper partitions. The quality of those partitions is comparable to the original structures, at the price of a more involved analysis. We then use those partitions to compute the counts and approximate the query ball as outlined at the beginning of this section. Second, we show that it is also possible to modify the algorithm so that the probability of including each qualifying point into the count is at least 1 -o (1) . This is of separate interest, as this yields the most efficient algorithm for approximate nearest neighbor search with space O ( nd ) , improving over [Kap15].

1.1 Related work

The problem of range counting when the query class Q is arbitrary, can be solved with an optimal error bound of √ n · poly(1 /ϵ, log |Q|·|U| δ ) [HR10]. In the setting of approximate near-neighbor queries, the size of Q and U can be thought of as at most exponential in the dimension d , by using a standard ϵ -net technique. Hence, their error is in fact smaller than ours when 4 c 2 ( c 2 +1) 2 ≥ 1 2 , or equivalently, when c ≤ 1 + √ 2 . However, whenever an approximation of c > 1 + √ 2 is acceptable, our error is smaller than [HR10]. Moreover, our approach is computationally efficient, whereas their method has runtime linear in the size of U , which is very inefficient.

When the query class is restricted to points in U , i.e., histogram queries, then the best achievable error is O ( 1 ϵ · min { log |U| , log(1 /δ ) } ) [Vad17]. When the points are in 1-dimensional Euclidean space taking values in [ u ] , and Q is a set of intervals, [DNPR10] got an ( ϵ, 0) -DP algorithm with error O ( 1 ϵ · log 1 . 5 u ) and a lower bound of Ω( log u ϵ ) . When resorting to ( ϵ, δ ) -DP, [BNSV15] showed an algorithm with error 2 (1+ o (1)) log ∗ u log(1 /δ ) /ϵ and a lower bound of Ω(log ∗ u · log(1 /δ ) /ϵ ) for e -ϵn/ log ∗ n ≤ δ ≤ 1 /n 2 . The problem for the axis-parallel rectangles have been further studied [CSS11, DNRR15] where polylogarithmic error bounds were obtained.

More generally, [MN12, NTZ13] showed an equivalence between the error and the discrepancy of the range space Q . Given that the discrepancy of many natural range spaces such as half-spaces, simplices, and spheres is n α ( d ) where α ( d ) ∈ [1 / 4 , 1 / 2] is a constant depending on d [Mat99], it rules out such a polylogarithmic error for these range spaces. For non-convex ranges, the discrepancy already becomes √ n reaching the threshold for arbitrary query ranges.

There are a number of papers based on space decompositions that provide a DP data structure for range counting that perform relatively well in practice [CPS + 12, HRMS09, LHMW14, QYL13a, QYL13b, XWG10, ZXX16] but perform poorly on high-discrepancy point sets.

Finally, we note that several prior works used LSH for differentially private kernel estimation [CS20, WNM23]. However, those works do not seem to be applicable to ball range queries, which correspond to uniform kernels.

2 Preliminaries

Range Counting: In the range counting problems, given a point set X ⊆ U , where U is a universe, the goal is to construct a differentially private data structure that for any query Q ⊆ U from a certain query family Q , one can compute | Q ∩ X | .

Here we focus on Near Neighbor counting queries in high dimensions. In particular U is a metric space equipped with a distance function dist . Given a prespecified radius parameter r , the query family Q = { B ( q, r ): q ∈ U} , where B ( q, r ) = { p ∈ U : dist( p, q ) ≤ r } is the ball of radius r around q . In other words, in this problem the goal is to count the number of neighbors of a given query point.

Similar to the work of [HY21], we consider the approximate variant of the problem where the points within distance r and cr of the query can be either counted or discarded. In particular for an approximation factor c ≥ 1 , any number between | X ∩ B ( q, r ) | and | X ∩ B ( q, cr ) | is valid.

Differential Privacy (DP): For two datasets X and X ′ , we use X ∼ X ′ to mean that they are neighboring datasets, i.e., one of them can be obtained from the other by an addition of a single element (in our problem a point).

Definition 2.1. (Differential Privacy [DR + 14]). For ϵ > 0 , δ ≥ 0 , a randomized algorithm A is ( ϵ, δ ) -differentially private ( ( ϵ, δ ) -DP) if for any two datasets X ∼ X ′ , and any possible outcome of the algorithm S ⊆ Range( A ), P [ A ( X ) ∈ S ] ≤ e ϵ · P [ A ( X ′ ) ∈ S ] + δ . When δ = 0 , the algorithm is said to have pure differential privacy.

The sensitivity of a function f is defined to be ∆ f = max X ∼ X ′ | f ( X ) -f ( X ′ ) | .

We use Lap( λ ) to denote the Laplace distribution with parameter λ with PDF P [ Z = z ] = 1 2 λ e -| z | /λ , which has mean 0 and variance λ 2 . We also use TLap(∆ , ϵ, δ ) to denote the Truncated Laplace distribution with PDF proportional to e -| z |· ϵ/ ∆ on the region [ -B,B ] , where B = ∆ ϵ · log ( 1 + e ϵ -1 2 δ ) .

Lemma 2.2 ((Truncated) Laplace Mechanism [DR + 14, GDGK20]) . Given a numeric function f that takes a dataset X as the input, and has sensitivity ∆ , the mechanism that outputs f ( X ) + Z where Z ∼ Lap(∆ /ϵ ) is ( ϵ, 0) -DP. In addition, if ϵ, δ ≤ 1 2 , f ( X ) + Z , where Z ∼ TLap(∆ , ϵ, δ ) , is ( ϵ, δ ) -DP. Moreover, the Truncated Laplace mechanism is always accurate up to error B .

3 Upper Bound

In this section, we describe the algorithm and perform the majority of the analysis for Theorem 1.1. We defer certain parts to Appendix A, and finish the proof there.

3.1 Setup

The algorithm we develop is inspired by the data-independent approximate nearest neighbor algorithm in [ALRW17]. We will use similar notation to their paper as well.

Define S d -1 to be the ( d -1) -dimensional unit sphere in R d . For any parameter r ∈ (0 , 2) , let α ( r ) = 1 -r 2 2 and β ( r ) = √ 1 -α ( r ) 2 be the cosine and the sine, respectively, of the angle between two points in S d -1 of distance r . For any parameter η > 0 , define

where u ∈ S d -1 is an arbitrary point on the unit sphere. Note that F ( η ) is independent of u by the spherical symmetry of Gaussians, and in fact equals the probability that a univariate standard Normal exceeds η . Next, for any parameters r ∈ (0 , 2) and η q , η u ∈ R , we define

where p, q ∈ S d -1 are points that have distance exactly r between them. As with F, G has no dependence on the specific points p, q but only on the distance r = ∥ p -q ∥ 2 between them. Note also that G ( r, η q , η u ) is non-increasing in r , even if η q or η u is negative.

Next, we note the following well-known bounds for F .

Algorithm 1 Data Structure

Proposition 3.1. For η ≥ 1 ,

In addition, as η →∞ , F ( η ) ≤ o (1) · e -η 2 / 2 .

3.2 Algorithm for data on the sphere

We now present an algorithm that obtains the desired privacy and accuracy guarantees, assuming both the data points and query points are on a unit sphere and the radius r is not too small. The initial algorithm description will not necessarily run in polynomial time, but we will later describe a simple modification to make this algorithm efficient. Finally, in Appendix A.2 we will modify the algorithm to work for general r and in Euclidean space, rather than just on the unit sphere. This follows from known embedding methods [BRS11], which is why we defer it to the Appendix.

Data Structure: We describe the data structure in words here, provide pseudocode in Algorithm 1. Fix K and T to be positive integers, to be set later. We generate a data structure, which is a T -ary tree of depth K . We index the levels as 0 , 1 , . . . , K , where level 0 is the root and level K consists of the T K leaves. For each node v except the root, we generate an independent d -dimensional standard Gaussian g v ∼ N (0 , I ) . In addition, we will define two parameters η q and η u , to be set later.

The tree nodes at level ℓ will partition the unit sphere S d -1 into T ℓ regions, plus an extra 'remainder' region of points that are not mapped into any of the T ℓ regions. At level 0 , all points in S d -1 are sent to the single region indexed by the root. For any node v in the tree of depth 0 ≤ ℓ ≤ K -1 and with children v 1 , . . . , v K , the region P v is partitioned into P v 1 , . . . , P v T 1 as follows. A point p ∈ P v is sent to P v i if i ≤ T is the smallest index such that ⟨ g v i , p ⟩ ≥ η u . Note that some points in P v may not be sent to any of P v 1 , . . . , P v T . For each leaf node v , we store an approximate count c v = | X ∩ P v | +TLap(1 , ϵ, δ ) , where we added Truncated Laplace noise. If the noised count c v is too small (at most ∆ ϵ · log ( 1 + e ϵ -1 δ ) ), we replace the count c v with 0 .

Answering a query: We describe the algorithm in words here, provide pseudocode in Algorithm 2. Given a query q , we 'send' q to every leaf node v = v K such that the path v 1 , v 2 , . . . , v K from the root v root satisfies ⟨ g v i , q ⟩ ≥ η q for all 1 ≤ i ≤ K . Hence, each query q will correspond to a subset V q of leaf nodes. The response to the query q will be

To improve the accuracy of the algorithm, we repeat this procedure O (log m ) times if there are m queries, each initialized to ( ϵ ′ , δ ′ ) -DP for ϵ ′ = ϵ/O (log m ) and δ ′ = δ/O (log m ) . The overall algorithm will still be ( ϵ, δ ) -DP, by the well-known Basic Composition theorem [DR + 14, Corollary 3.15]. For each query, we output the median response of the individual data structure responses. In the analysis, we will ignore this amplification procedure, as by a Chernoff bound it suffices to show that each individual query is answered accurately with at least 2 / 3 probability.

3.3 Analysis

Privacy: The analysis of privacy is quite simple. First, note that the responses to all of the queries do not depend on the data directly, but only depend on the counts { c v } . If we let ˆ c v ( X ) := | X ∩ P v | and ˜ c v ( X ) := | X ∩ P v | +TLap(1 , ϵ, δ ) , then note that c v ( X ) is entirely dependent on ˜ c v ( X ) . Hence, it suffices to show the following.

Lemma 3.2 (Privacy) . The set { ˜ c v ( X ) } v leaf ∈T is ( ϵ, δ ) -DP with respect to the data X . Hence, the entire algorithm is also ( ϵ, δ ) -DP, even if we have an arbitrary number of queries.

Proof. Consider two adjacent datasets X,X ′ , i.e., where we either removed or added a single data point from X to obtain X ′ . Note that the construction partitioning of the dataset into leaves is not dependent on the data, but merely on random vectors g v over all nodes v . Therefore, if we condition on the partitioning of the entire space, the values ˆ c v ( X ) = | X ∩ P v | and ˆ c v ( X ′ ) = | X ′ ∩ P v | are the same for all but at most one leaf v , which changes by at most 1 . The sensitivity of each such ˆ c v ( X ) is at most 1 , which means that we have ( ϵ, δ ) -privacy loss from ˜ c v ( X ) . Because this happens for a single choice of v , we have ( ϵ, δ ) -privacy loss in total.

Accuracy: We will only consider the accuracy with respect to a fixed query q and a single copy of the data structure, and show accuracy holds in expectation. For a set of m queries, since we repeat this data structure O (log m ) times and use the median estimate, a Chernoff bound implies all queries will be answered accurately with at least 99% probability.

We prove the following lemma to bound accuracy.

Lemma 3.3 (Accuracy, assuming parameters are set properly) . Suppose the following hold, for some fixed choice of parameters c, ρ, K, T, η q , η u :

( T · F ( η q )) K ≤ n ρ + o (1) . (Additive error due to Laplace noise).

e -TF ( η u ) = o (1 /K ) , and G ( r,η q ,η u ) F ( η u ) = 1 -o (1 /K ) . (Multiplicative error due to not including points within r of q ).

Then, if ans is the response when querying a fixed q on a dataset X , with probability at least 2 / 3 , ans is between and

In other words, we solve the ( c, r ) -near neighbor counting problem with probability at least 2 / 3 , up to a multiplicative 1 ± o (1) factor and an additive O ( 1 ϵ · log 1 δ · n ρ + o (1) ) factor.

In Subsection 3.4, we will show how to set T, K, η q , η u so that the three conditions above hold.

Proof. Our error comes from two sources. The first is the Truncated Laplace noise that we add, which we add in | V q | locations to compute the answer to query q . This adds an additive noise of at most O ( 1 ϵ · log 1 δ · E | V q | ) in expectation, as | c v -| X ∩ P v || ≤ O ( 1 ϵ · log 1 δ ) with probability 1 .

To bound E | V q | , note that it simply equals the sum of the probabilities that q ∈ P v for each leaf v, which is T K times the probability that q ∈ P v for a fixed v . This probability is just F ( η q ) K since each Gaussian along the path to v is independent. Hence, the first source error has magnitude at most O ( 1 ϵ · log 1 δ · ( T · F ( η q )) K ) ≤ O ( 1 ϵ · log 1 δ · n ρ + o (1) ) in expectation.

Hence, if we let ̂ ans be the response we would have received if we did not add Laplace noise (i.e., if we used ˆ c v instead of c v ), then E [ | ans -̂ ans | ] ≤ O ( 1 ϵ · log 1 δ · n ρ + o (1) ) , so | ans -̂ ans | ≤ O ( 1 ϵ · log 1 δ · n ρ + o (1) ) with probability at least 0 . 9 by Markov's inequality.

The second source of error is that ̂ ans counts the number of points in X that are in P v for some v ∈ V q , whereas we actually want to count the number of points in X that are within distance r of q (tolerating the inclusion of points up to distance cr of q ). Hence, we need to make sure of two things, corresponding to not underestimating or overestimating, respectively:

Most points in X within distance r of q are mapped to some P v where v ∈ V q .

Few points in X that are not within distance cr of q are mapped to some P v where v ∈ V q .

This way, we will show that with high probability,

which is sufficient.

For any point p of distance s from q , we compute the probability that p ∈ P v for some v ∈ V q . Let's start by assuming K = 1 , i.e., we have a depth-1 tree with leaves labeled 1 , 2 , . . . , T . In this case there are two possibilities for failure: either p is not mapped to any P i for 1 ≤ i ≤ T , or p is mapped to some P i but q is not sent there. Suppose p is mapped to P 1 : this means ⟨ g 1 , p ⟩ ≥ η u . Conditioned on this event, the probability that ⟨ g 1 , q ⟩ ≥ η q is precisely P [ ⟨ g 1 , q ⟩ ≥ η q |⟨ g 1 , p ⟩ ≥ η u ] = P [ ⟨ g 1 ,q ⟩≥ η q , ⟨ g 1 ,p ⟩≥ η u ] P [ ⟨ g 1 ,p ⟩≥ η u ] = G ( s,η q ,η u ) F ( η u ) . Suppose p is mapped to P i for i ≥ 2 . Then, the probability that ⟨ g i , q ⟩ ≥ η q is P [ ⟨ g i , q ⟩ ≥ η q |⟨ g i , p ⟩ ≥ η u , ⟨ g j , p ⟩ < η u ∀ j < i ] . But even if p, q are fixed, then the values of ⟨ g j , p ⟩ for j < i are independent of ( ⟨ g i , p ⟩ , ⟨ g i , q ⟩ ) . Hence, we may remove the conditioning on ⟨ g j , p ⟩ < η u ∀ j < i , to again say that the conditional probability is G ( s,η q ,η u ) F ( η u ) . Hence, the probability that p ∈ P v for v ∈ V q , conditioned on p being in some P v , is G ( s,η q ,η u ) F ( η u ) . The probability that s ̸∈ P v for any v is (1 -F ( η u )) T , so the overall probability that p ∈ P v for v ∈ V q , for K = 1 , is

Note that 1 -e -TF ( η u ) ≤ 1 -(1 -F ( η u )) T ≤ 1 .

For general depths, note that the Gaussians selected at each level are independent of the previous levels. Hence, the success probability simply raises to the power of K , or equals

Note that 1 ≥ (1 -e -TF ( η u ) ) K ≥ 1 -K · e -TF ( η u ) . Hence, the probability that a point p of distance s from q is mapped to some P v where v ∈ V q is between ( 1 -K · e -TF ( η u ) ) · ( G ( s,η q ,η u ) F ( η u ) ) K and ( G ( s,η q ,η u ) F ( η u ) ) K .

Because G ( s, η q , η u ) is an increasing function in s , the probability at any point of distance s ≤ r from q is not included in the count ̂ ans is at least ( 1 -K · e -TF ( η u ) ) · ( G ( r,η q ,η u ) F ( η u ) ) K = 1 -o (1) , as we are assuming that e -T · F ( η u ) = o (1 /K ) and G ( r,η q ,η u ) F ( η u ) = 1 -o (1 /K ) . Thus, by Markov's inequality, with probability at least 0 . 9 , ̂ ans ≥ (1 -o (1)) · | X ∩ B 2 ( q, r ) | . Next, the probability that any point of distance s ≥ cr from q is included in the count ̂ ans is at most ( G ( cr,η q ,η u ) F ( η u ) ) K , which we assume is at most n -1+ ρ + o (1) . Because there are at most n points, by Markov's inequality, ̂ ans ≤ | X ∩ B 2 ( q, cr ) | + n ρ + o (1) with probability at least 0 . 9 .

Runtime: While we have written the algorithm to create a T K -sized tree, we can speed up the implementation by only including the necessary parts of the data structure.

We can bound the runtime as follows. Naively, for preprocessing, it takes space and time O ( T K · d ) to construct the tree and generate and store the Gaussians. Next, for each point p , if p ∈ P v , it takes up to O ( T · d ) time to determine which child of v is that p will be sent to. Since p is sent to only one node in each partition, this takes total time O ( K · T · d ) per point, which means O ( n · K · T · d ) total time to partition the points in X . To improve the preprocessing time, we do not generate the full tree: rather, we only generate a node of the tree (with a corresponding Gaussian g v ) if we need to check whether some data point p ∈ X is sent to the node, which means we can improve the preprocessing time to O ( n · K · T · d ) , without the extra term of O ( T K · d ) . Note that if some leaf node v was not created in our modified implementation, then no point would have been sent to some partition piece P v , which means | X ∩ P v | +TLap(1 , ϵ 2 , δ 2 ) ≤ 2 ϵ · log ( 1 + e ϵ/ 2 -1 δ ) . So, we would have set c v to be 0 anyway, which means that this modification does not affect any of the responses to queries.

To answer a query q , if q ∈ P v (where v is possibly not a leaf node) it takes O ( T · d ) time to determine all of the children of v that q will be sent to. In expectation, q is sent to ( TF ( η q )) ℓ nodes at level ℓ, which means at level ℓ +1 we need to check up to ( TF ( η q )) ℓ · T nodes in expectation. (If a node w is not created in our implementation, we do not have to check it, since we know c v = 0 for any leaf v that is a descendant of w .) Therefore, the total time it takes to determine all leaf nodes q is sent to, in expectation, is O ( d · ∑ K -1 ℓ =0 ( TF ( η q )) ℓ · T ) = O (( TF ( η q )) K · K · T · d ) . Finally, we can add c v over all leaf nodes v that q is sent to, which takes an additional O (( TF ( η q )) K ) time in expectation. Hence, we have the following lemma.

Lemma 3.4 (Runtime) . The total preprocessing time of the data structure is O ( n · K · T · d ) , and the expected time needed to answer each query is O (( TF ( η q )) K · K · T · d ) .

3.4 Parameter Settings and Finishing the Accuracy/Runtime Analysis

In this section, we set parameters properly so that Lemmas 3.3 and 3.4 match the goals of Theorem 1.1. We recall that c > 1 is a fixed constant representing the approximation ratio. We assume that d = (log n ) O (1) and r = Θ ( 1 (log n ) 1 / 8 ) . For fixed c and r as set above, we set the parameters as follows:

K := √ ln n .

η u := √ ln n K · λ r , for some constant λ = λ ( c ) , that we will set later.

η q := α ( r ) · η u -2 β ( r ) · √ ln K . Recall that α ( r ) = 1 -r 2 2 and β ( r ) = √ 1 -α ( r ) 2 .

T := 10 ln K/F ( η u ) .

First, we must show that our parameter choices imply that the algorithm is accurate. We prove the following lemma, for which the proof is deferred to Appendix A.

√

Lemma 3.5 (Accuracy, completed) . For an appropriate choice of λ ( c ) = 2 2 c c 2 +1 , all three conditions in Lemma 3.3 hold, with ρ = 4 c 2 ( c 2 +1) 2 = O ( 1 c 2 ) . Hence, Lemma 3.3 holds without the conditions, for ρ = 4 c 2 ( c 2 +1) 2 .

Finally, we show that the runtime is good, under our parameter choices.

Lemma3.6 (Runtime, completed) . For the parameters we have defined, assuming r = Θ ( 1 (log n ) 1 / 8 ) , the total preprocessing time is n 1+ o (1) · d and the expected time to answer each query is n ρ + o (1) · d , for ρ = 4 c 2 ( c 2 +1) 2 .

Proof. The preprocessing time is O ( n · K · T · d ) . We know that T = 10 ln K F ( η u ) , and that 1 F ( η u ) ≤ Ω( η u ) · e η 2 u / 2 ≤ (log n ) O (1) · e ln n · λ 2 / ( r 2 · K ) . By our settings of r = Θ((log n ) -1 / 8 ) and K = √ ln n , and since λ is a constant, this equals n Θ(1 / (log n ) 1 / 4 ) = n o (1) . Hence, 1 /F ( η u ) = n o (1) , and since K = O ( √ log n ) , the total preprocessing time is n 1+ o (1) · d .

The time to answer each query is O (( TF ( η q )) K · K · T · d ) . By Lemma 3.5, we know that ( TF ( η q )) K = n ρ + o (1) . Moreover, we know that K = (log n ) O (1) , and we already saw that T = 10ln K/F ( η u ) = n o (1) . Hence, the time to answer each query is n ρ + o (1) · d .

Lemmas 3.2, 3.5, and 3.6 will end up being sufficient to prove Theorem 1.1 if we assume the data lies on a d = (log n ) O (1) -dimensional unit sphere, and if r = Θ((log n ) 1 / 8 ) . To finish the analysis, we must show that this assumption can be made without loss of generality, which will follow from an embedding argument of [BRS11]. We defer this argument, and the proof of the remainder of Theorem 1.1, to Appendix A.2.

Remark. We note that the data structure can be implemented so that extra space is only O ( nd ) . Instead of generating and storing the Gaussians g v independently, we note that in each level ℓ , each group of T siblings can reuse exactly the same random set of T Gaussians. The analysis goes through mutatis mutandis as independence between different branches is not needed.

Acknowledgements: PI was supported by the NSF TRIPODS program (award DMS-2022448), Simons Investigator Award, and GIST-MIT Research Collaboration grant.

SN is supported by a NSF Graduate Fellowship (Grant No. 1745302) and a Google PhD Fellowship. AA supported in part by NSF (CCF2008733) and ONR (N00014-22-1-2713).

References

[Abo18] John M Abowd. The us census bureau adopts differential privacy. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining , pages 2867-2867, 2018.

[AI06] Alexandr Andoni and Piotr Indyk. Near-optimal hashing algorithms for approximate nearest neighbor in high dimensions. pages 459-468, 2006.

[ALRW17] Alexandr Andoni, Thijs Laarhoven, Ilya P. Razenshteyn, and Erik Waingarten. Optimal hashing-based time-space trade-offs for approximate near neighbors. In Philip N. Klein, editor, Proceedings of the Twenty-Eighth Annual ACM-SIAM Symposium on Discrete Algorithms (SODA) , pages 47-66. SIAM, 2017.

[BEM + 17] Andrea Bittau, Úlfar Erlingsson, Petros Maniatis, Ilya Mironov, Ananth Raghunathan, David Lie, Mitch Rudominer, Ushasree Kode, Julien Tinnes, and Bernhard Seefeld.

Prochlo: Strong privacy for analytics in the crowd. In Proceedings of the 26th symposium on operating systems principles , pages 441-459, 2017.

[BNSV15] Mark Bun, Kobbi Nissim, Uri Stemmer, and Salil Vadhan. Differentially private release and learning of threshold functions. In 2015 IEEE 56th Annual Symposium on Foundations of Computer Science , pages 634-649. IEEE, 2015.

[BRS11] Yair Bartal, Ben Recht, and Leonard J. Schulman. Dimensionality reduction: Beyond the johnson-lindenstrauss bound. In Proceedings of the Twenty-Second Annual ACM-SIAM Symposium on Discrete Algorithms (SODA) , pages 868-887. SIAM, 2011.

[Chr17] Tobias Christiani. A framework for similarity search with space-time tradeoffs using locality-sensitive filtering. In Proceedings of the Twenty-Eighth Annual ACM-SIAM Symposium on Discrete Algorithms , pages 31-46. SIAM, 2017.

[CL01] Bernard Chazelle and Alexey Lvov. The discrepancy of boxes in higher dimension. Discret. Comput. Geom. , 25(4):519-524, 2001.

[CPS + 12] Graham Cormode, Cecilia Procopiuc, Divesh Srivastava, Entong Shen, and Ting Yu. Differentially private spatial decompositions. In 2012 IEEE 28th International Conference on Data Engineering , pages 20-31. IEEE, 2012.

[CS20] Benjamin Coleman and Anshumali Shrivastava. A one-pass private sketch for most machine learning tasks. arXiv preprint arXiv:2006.09352 , 2020.

[CSS11] T-H Hubert Chan, Elaine Shi, and Dawn Song. Private and continual release of statistics. ACM Transactions on Information and System Security (TISSEC) , 14(3):1-24, 2011.

[DKM + 06] Cynthia Dwork, Krishnaram Kenthapadi, Frank McSherry, Ilya Mironov, and Moni Naor. Our data, ourselves: Privacy via distributed noise generation. In Advances in Cryptology-EUROCRYPT 2006: 24th Annual International Conference on the Theory and Applications of Cryptographic Techniques, St. Petersburg, Russia, May 28-June 1, 2006. Proceedings 25 , pages 486-503. Springer, 2006.

[DKY17] Bolin Ding, Janardhan Kulkarni, and Sergey Yekhanin. Collecting telemetry data privately. Advances in Neural Information Processing Systems , 30, 2017.

[DMNS06] Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise to sensitivity in private data analysis. In Theory of Cryptography: Third Theory of Cryptography Conference, TCC 2006, New York, NY, USA, March 4-7, 2006. Proceedings 3 , pages 265-284. Springer, 2006.

[DMNS16] Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise to sensitivity in private data analysis. Journal of Privacy and Confidentiality , 7(3):17-51, 2016.

[DNPR10] Cynthia Dwork, Moni Naor, Toniann Pitassi, and Guy N Rothblum. Differential privacy under continual observation. In Proceedings of the forty-second ACM symposium on Theory of computing , pages 715-724, 2010.

[DNRR15] Cynthia Dwork, Moni Naor, Omer Reingold, and Guy N Rothblum. Pure differential privacy for rectangle queries via private partitions. In Advances in Cryptology-ASIACRYPT 2015: 21st International Conference on the Theory and Application of Cryptology and Information Security, Auckland, New Zealand, November 29-December 3, 2015, Proceedings, Part II , pages 735-751. Springer, 2015.

[DR + 14] Cynthia Dwork, Aaron Roth, et al. The algorithmic foundations of differential privacy. Foundations and Trends® in Theoretical Computer Science , 9(3-4):211-407, 2014.

[EPK14] Úlfar Erlingsson, Vasyl Pihur, and Aleksandra Korolova. Rappor: Randomized aggregatable privacy-preserving ordinal response. In Proceedings of the 2014 ACM SIGSAC conference on computer and communications security , pages 1054-1067, 2014.

[G + 16] Andy Greenberg et al. Apple's 'differential privacy' is about collecting your data-but not your data. Wired, June , 13:1, 2016.

[GDGK20] Quan Geng, Wei Ding, Ruiqi Guo, and Sanjiv Kumar. Tight analysis of privacy and utility tradeoff in approximate differential privacy. In The 23rd International Conference on Artificial Intelligence and Statistics (AISTATS) , volume 108 of Proceedings of Machine Learning Research , pages 89-99. PMLR, 2020.

[HKR12] Justin Hsu, Sanjeev Khanna, and Aaron Roth. Distributed private heavy hitters. In Automata, Languages, and Programming: 39th International Colloquium, ICALP 2012, Warwick, UK, July 9-13, 2012, Proceedings, Part I 39 , pages 461-472. Springer, 2012.

[HR10] Moritz Hardt and Guy N Rothblum. A multiplicative weights mechanism for privacypreserving data analysis. In 2010 IEEE 51st annual symposium on foundations of computer science , pages 61-70. IEEE, 2010.

[HRMS09] Michael Hay, Vibhor Rastogi, Gerome Miklau, and Dan Suciu. Boosting the accuracy of differentially-private histograms through consistency. arXiv preprint arXiv:0904.0942 , 2009.

[HY21] Ziyue Huang and Ke Yi. Approximate range counting under differential privacy. In 37th International Symposium on Computational Geometry (SoCG 2021) . Schloss Dagstuhl-Leibniz-Zentrum für Informatik, 2021.

[IM98] Piotr Indyk and Rajeev Motwani. Approximate nearest neighbors: towards removing the curse of dimensionality. In Proceedings of the thirtieth annual ACM symposium on Theory of computing , pages 604-613, 1998.

[Kap15] Michael Kapralov. Smooth tradeoffs between insert and query complexity in nearest neighbor search. pages 329-342, New York, NY, USA, 2015. ACM.

[LHMW14] Chao Li, Michael Hay, Gerome Miklau, and Yue Wang. A data-and workload-aware algorithm for range queries under differential privacy. arXiv preprint arXiv:1410.0265 , 2014.

[LJW + 07] Qin Lv, William Josephson, Zhe Wang, Moses Charikar, and Kai Li. Multi-probe lsh: efficient indexing for high-dimensional similarity search. In VLDB , 2007.

[LN14] Huy Le Nguyen. Algorithms for high dimensional data . PhD thesis, Princeton University, 2014.

[Mat99] Jiˇ rí Matoušek. Geometric discrepancy: An illustrated guide. 1999.

[MN04] Manor Mendel and Assaf Naor. Euclidean quotients of finite metric spaces. Advances in Mathematics , 189(2):451-494, 2004.

[MN12] S. Muthukrishnan and Aleksandar Nikolov. Optimal private halfspace counting via discrepancy. In Proceedings of the 44th Symposium on Theory of Computing Conference (STOC) , pages 1285-1292. ACM, 2012.

[NTZ13] Aleksandar Nikolov, Kunal Talwar, and Li Zhang. The geometry of differential privacy: the sparse and approximate cases. In Proceedings of the forty-fifth annual ACM symposium on Theory of computing , pages 351-360, 2013.

[Pan06] Rina Panigrahy. Entropy based nearest neighbor search in high dimensions. In Proceedings of the seventeenth annual ACM-SIAM symposium on Discrete algorithm , pages 1186-1195, 2006.

[QYL13a] Wahbeh Qardaji, Weining Yang, and Ninghui Li. Differentially private grids for geospatial data. In 2013 IEEE 29th international conference on data engineering (ICDE) , pages 757-768. IEEE, 2013.

[QYL13b] Wahbeh Qardaji, Weining Yang, and Ninghui Li. Understanding hierarchical methods for differentially private histograms. Proceedings of the VLDB Endowment , 6(14):19541965, 2013.

[Sha14] Stephen Shankland. How google tricks itself to protect chrome user privacy. CNET, October , 2014.

[Vad17] Salil Vadhan. The complexity of differential privacy. Tutorials on the Foundations of Cryptography: Dedicated to Oded Goldreich , pages 347-450, 2017.

[WNM23] Tal Wagner, Yonatan Naamad, and Nina Mishra. Fast private kernel density estimation via locality sensitive quantization. 2023.

[XWG10] Xiaokui Xiao, Guozhang Wang, and Johannes Gehrke. Differential privacy via wavelet transforms. IEEE Transactions on knowledge and data engineering , 23(8):1200-1214, 2010.

[ZXX16] Jun Zhang, Xiaokui Xiao, and Xing Xie. Privtree: A differentially private algorithm for hierarchical decompositions. In Proceedings of the 2016 international conference on management of data , pages 155-170, 2016.

A Omitted Proofs from Section 3

A.1 Completion of Accuracy Analysis

Here, we complete the accuracy analysis by proving Theorem 3.5.

Proof of Theorem 3.5. To bound ( T · F ( η q )) K , note that (10 ln K ) K ≤ n o (1) by our setting of K = √ ln n . Therefore, it suffices to show that ( F ( η q ) /F ( η u )) K ≤ n ρ + o (1) . However, by Proposition 3.1, we can upper bound F ( η q ) /F ( η u ) ≤ O ( η u ) · e ( η 2 u -η 2 q ) / 2 . But we know that η u ≤ (log n ) O (1) , which implies that O ( η u ) K ≤ n o (1) . Hence, it suffices to bound ( e ( η 2 u -η 2 q ) / 2 ) K . Indeed, we can write

Above, the third line follows from the fact that 1 -α ( r ) 2 ≤ r 2 and α ( r ) · β ( r ) ≤ β ( r ) ≤ r , and the fourth line follows from our definition of K . Therefore,

Next, note that e -TF ( η u ) = e -10 log K = K -10 = o (1 /K ) , since K = ω (1) .

Next, we bound G ( r,η q ,η u ) F ( η u ) . Since G ( r, η q , η u ) = P [ ⟨ g, q ⟩ ≥ η q and ⟨ g, p ⟩ ≥ η u ] for ∥ p -q ∥ 2 = r, we can rewrite this as G ( r,η q ,η u ) F ( η u ) = P [ ⟨ g, q ⟩ ≥ η q |⟨ g, p ⟩ ≥ η u ] for ∥ p -q ∥ 2 = r. Since p, q are unit vectors, we can write q = α ( r ) · p + β ( r ) · p ′ , where p ′ is a unit vector orthogonal to p . Hence, if we let Y = ⟨ g, p ⟩ and Z = ⟨ g, p ′ ⟩ , then Y, Z are independent standard Gaussians and our goal is to bound P [ α ( r ) · Y + β ( r ) · Z ≥ η q | Y ≥ η u ] . But, η q = α ( r ) · η u -2 β ( r ) · √ ln K, which means if Z ≥ -2 √ ln K , then Y ≥ η u automatically implies α ( r ) · Y + β ( r ) · Z ≥ η q . Therefore, our probability is at least P ( Z ≥ -2 √ ln K ) = 1 -P ( Z ≥ 2 √ ln K ) = 1 -o (1 /K ) , since P ( Z ≥ 2 √ ln K ) = O (1 /K 2 ) .

Finally, we must bound ( G ( cr,η q ,η u ) F ( η u ) ) K . Note that G ( cr,η q ,η u ) F ( η u ) = P [ ⟨ g, q ⟩ ≥ η q |⟨ g, p ⟩ ≥ η u ] for ∥ p -q ∥ 2 = cr , which means q = α ( cr ) · p + β ( cr ) · p ′ . Again, letting Y = ⟨ g, p ⟩ and Z = ⟨ g, p ′ ⟩ , we wish to bound

We can rewrite this as

If we write W = Y -η u , note that the PDF of W is 1 √ 2 π · e -( W + η u ) 2 / 2 . The probability that W ≥ 0 is between Ω ( 1 η u ) · e -η 2 u / 2 and e -η 2 u / 2 , which means

Next, a simple calculation tells us that

and

The CDF of W cr ′ conditioned on W ≥ 0 is bounded above by O ( cr ′ · η u ) · e -cr ′ · η u · w ≤ (log n ) O (1) · e -S · w , where S = c · r ′ r · λ · √ ln n/K . In addition, we write p Z to represent the PDF of Z (which is a standard Gaussian) and T = √ ln n/K · ( c 2 -1) λ/ (2 c ) · (1 -o (1)) . Note that S > T , since it suffices for c > c 2 -1 2 c · (1 + o (1)) , which is true for any constant c . Then, we have that (3) is at most

The third line uses the fact that ( S -T ) w ≥ 0 when w ≥ 0 (since S ≥ T ) and that Tw ≤ 0 when w ≤ 0 (since T is positive as we assume c ≥ 1 ). Therefore,

To finish, we will want to set λ 2 2 = ρ and ( c 2 -1) 2 λ 2 8 c 2 = 1 -ρ . Solving gives us λ = 2 √ 2 c c 2 +1 and ρ = 4 c 2 ( c 2 +1) 2 .

A.2 Embedding into the Euclidean sphere

In the previous sections, we assumed that the data lied on a reasonably low-dimensional Euclidean sphere, and that r was some reasonably small (but not too small) parameter. In this section, we explain why this assumption can be made without loss of generality, by using a previous work of [BRS11] which allows us to embed the data into a Euclidean sphere.

Lemma A.1. [BRS11, Lemma 6, rephrased] Fix integers d, d ′ ≥ 1 and a parameter 0 < γ < 1 2 . There exists a randomized map Θ : R d → S d ′ -1 , where S d ′ -1 represents the unit sphere in R d ′ , that can be computed in time O ( d · d ′ ) for any point x ∈ R d , with the following properties for any x, y ∈ R d .

We will set d ′ = O ( log n γ 2 ) , where we recall that n equals the number of data points and m equals the number of queries. Then, if we generate the randomized map Θ ahead of time, with high probability the conditions in Lemma A.1 hold for all n data points and any fixed query point. (In the final algorithm, we repeat the embedding O (log m ) times to amplify the success probability).

Corollary A.2. Suppose that ( cr ) 2 ≤ γ 2 , and d ′ = C · log n γ 2 for some sufficiently large constant C . Then, with at least 0 . 99 probability over the randomized map Θ : R d → S d ′ -1 , for all x ∈ X and any fixed query y , ∥ x -y ∥ 2 ≤ r implies ∥ Θ( x ) -Θ( y ) ∥ 2 ≤ (1 + γ ) · r and ∥ x -y ∥ 2 ≥ cr implies ∥ Θ( x ) -Θ( y ) ∥ 2 ≥ (1 -γ ) · cr .

Proof. We assume that Θ was chosen so that for every x ∈ X and for the fixed y , the three claims in Lemma A.1 all hold.

Fix some pair ( x, y ) , where x ∈ X,y ∈ Q . First, if ∥ x -y ∥ 2 ≤ r , then since r ≤ √ γ , ∥ Θ( x ) -Θ( y ) ∥ 2 ≤ (1+ γ ) · ∥ x -y ∥ 2 = (1+ γ ) · r . Next, if c · r ≤ ∥ x -y ∥ 2 ≤ √ γ , then ∥ Θ( x ) -Θ( y ) ∥ 2 ≥ (1 -γ ) ·∥ x -y ∥ 2 ≥ (1 -γ ) · cr . Otherwise, if ∥ x -y ∥ 2 ≥ √ γ , then ∥ Θ( x ) -Θ( y ) ∥ 2 ≥ √ γ 2 ≥ c · r .

We can now finish the proof of Theorem 1.1.

Proof. We will prove that we can solve the ( c · 1+ γ 1 -γ , r 1+ γ ) -near neighbor counting with differential privacy, where r = (log n ) -1 / 8 and γ = (log n ) -1 / 8 , over Euclidean space. By scaling, this automatically implies the same result for any choice of r , and c replaced with c (1+ γ ) 1 -γ = c (1 + o (1)) .

We will run O (log m ) independent copies of the data structure and algorithm, where each individual data structure satisfies ( ϵ ′ , δ ′ ) = ( ϵ O (log m ) , δ O (log m ) ) -DP. By basic composition, this implies ( ϵ, δ ) -DP overall. For each query, we output the median of the responses of each data structure, which by a Chernoff bound will significantly decrease the failure probability.

For each copy of the data structure, we first preprocess the data X ∈ R d by using the random map Θ from Lemma A.1 to map each point x ∈ X to Θ( x ) , which takes O ( nd · d ′ ) time. Next, we create the data structure on Θ( X ) , which is on a d ′ = O ( log n γ 2 ) = (log n ) O (1) -dimensional sphere. To answer any query q ∈ R d , we compute Θ( q ) in O ( d · d ′ ) time, and answer the query Θ( q ) using the data structure created on Θ( X ) . By Corollary A.2, with 0 . 99 probability, any accurate answer for ( c, r ) -near neighbor counting on S d ′ -1 for a fixed query Θ( q ) is an accurate answer for ( ( cr ) / (1 -γ ) r/ (1+ γ ) , r 1+ γ ) = ( c · 1+ γ 1 -γ , r 1+ γ ) -near neighbor counting problem for q over R d . Moreover, the embedding Θ is oblivious to the data, which means privacy is preserved.

Now, we have embedded the data and queries onto the unit sphere S d ′ -1 for d ′ = (log n ) O (1) , in O ( nd · d ′ ) = ˜ O ( nd ) preprocessing time. We may now create the data structure (Algorithm 1) and answer each query (Algorithm 2) on S d ′ -1 . We recall that, to improve the failure probability so that every query is answered with high probability, we replaced ( ϵ, δ ) with ( ϵ ′ , δ ′ ) = ( ϵ O (log m ) , δ O (log m ) ) , and we will take the median over O (log m ) independent copies of the data structure, for each query. By Lemmas 3.2 and 3.6, the privacy and runtime conditions of Theorem 1.1 are all satisfied. Moreover, by Lemma 3.5, with probability at least 2 / 3 -0 . 01 ≥ 0 . 6 , for each query q , each copy of the data structure is accurate, replacing the additive error 1 ϵ log 1 δ · n ρ + o (1) with log m ϵ log log m δ · n ρ + o (1) . By a Chernoff bound, the overall algorithm (taking the median of O (log m ) copies) is accurate on each

query with probability at least 1 -1 100 m , so it is accurate on all queries simultaneously with at least 0 . 99 probability. This concludes the proof.

B Lower Bound

In this section, we prove Theorem 1.2.

Given n points z 1 , . . . , z n ∈ { 0 , 1 } d , and a series of m queries B 1 , . . . , B m represented by axisparallel boxes, we create the matrix A i,j to be the indicator of B i containing z j . We define the red-blue discrepancy to be max x ∈{-1 , 1 } n ∥ Ax ∥ ∞ .

First, we have the following result due to Chazelle and Lvov [CL01].

Theorem B.1. [CL01] For sufficiently large n and d = C log n for a large constant C , there exists a set of n points z 1 , . . . , z n ∈ { 0 , 1 } d and n boxes B 1 , . . . , B n such that the matrix { A i,j } ∈ R n × n has red-blue discrepancy n Ω(1) = 2 Ω( d ) .

Now, we switch to the result of Muthukrishnan and Nikolov [MN12]. They have a slighly more general definition of discrepancy, of disc p,α ( A ) = min x ∈{-1 , 0 , 1 } d , ∥ x ∥ 1 ≥ α · d ∥ Ax ∥ p , so the above result is written as a lower bound on disc( A ) := disc ∞ , 1 ( A ) . Now, we describe their theorem.

Theorem B.2. [MN12] For all x ∈ { 0 , 1 } n , there exist constants ϵ, δ > 0 such that no mechanism M = {M n } that satisfies

is ( ϵ, δ ) -DP, where two data points x, x ′ ∈ { 0 , 1 } n are adjacent if they differ exactly in one coordinate location.

Given these lemmas, we are ready to prove our main lower bound, Theorem 1.2.

Proof of Theorem 1.2. We set m = n and d = C log n for some constant C . By using Theorem B.1, we can construct a public set { z 1 , . . . , z n } of distinct points in { 0 , 1 } d and a fixed set of boxes B 1 , . . . , B n , that generate a matrix A with discrepancy disc( A ) ≥ n ρ , where ρ > 0 is a fixed constant. For any x ∈ { 0 , 1 } n , we define the dataset D x will be the subset of the public set where z i ∈ D x if and only if x i = 1 . Then, Ax would be an m = n dimensional vector that precisely tells us the number of points in D x are in each box. In addition, if x, x ′ are adjacent in { 0 , 1 } n , then D x , D x ′ are adjacent datasets. Next for any axis-parallel box B i , we create a query point q i ∈ { 0 , 1 } d as follows. In the k th coordinate dimension, if B i intersects either both or neither of the planes x k = 0 and x k = 1 , we set q i,k = 1 / 2 . Otherwise, if B i only intersects the plane x k = 0 , we set q i,k = -1 / 2 . Finally, if B i only intersects the plane x k = 1 , we set q i,k = 3 / 2 .

Suppose that A is an ( ϵ, δ ) -DP algorithm acting on subsets S of { 0 , 1 } d of size at most n , that returns an estimate to | S ∩ B ∞ ( q i , 1) | for all queries q i . Since every q i has half-integral coordinates, and every data point has integral coordinates, we have that B ∞ ( q i , 1) = B ∞ ( q i , 1 / 2) = B ∞ ( q i , 3 / 2 -α ) for any α > 0 , and these all equal B i . So, for any subset S ⊂ { 0 , 1 } d ,

Suppose that for any S ⊂ { 0 , 1 } d of size at most n , with probability at least 2 / 3 , the output of A ( S ) for each query q i differs by at most T from | S ∩ B i | . Then, we can consider the algorithm M that acts on an element in { 0 , 1 } n , where M ( x ) = A ( D x ) . Since A is ( ϵ, δ ) -DP, this implies that M is also ( ϵ, δ ) -DP, with respect to adjacent elements in { 0 , 1 } n . Moreover, ( A · x ) i = ∑ j A i,j x j = |{ j : A i,j = x j = 1 }| = | D x ∩ B i | . Hence, by Theorem B.2, there must exist x ∈ { 0 , 1 } n such that

so T ≥ disc( A ) . This completes the proof as disc( A ) ≥ n ρ

2 .

C Extensions

ℓ 1 balls. It is easy to extend our algorithms to ranges that are balls in the ℓ 1 norm. This relies on the well-known fact that the ℓ 1 norm embeds (almost) isometrically into a negative-type metric, i.e., into a squared Euclidean metric ( ℓ 2 ) 2 . We use the following constructive version of this fact, from [LN14, Theorem 116] based on [MN04].

Theorem C.1 ([LN14] 4 ) . Let 1 ≤ p < 2 . Let X ⊂ R d be a point set with ℓ p -aspect ratio Φ . There is a mapping f : X → R d ′ for d ′ = d · poly(log Φ , log d, 1 /ϵ ) such that for every x, y ∈ X ,

The above theorem, applied for p = 1 , reduces approximate range queries for ℓ 1 balls with approximation factor c to approximate range queries for ( ℓ 2 ) 2 balls with approximation factor c/ (1 + ϵ ) and d replaced by d ′ . The latter problem is equivalent to approximate range queries for ℓ 2 balls with approximation factor √ c/ (1 + ϵ ) . The final bound follows by following this sequence of reductions.

Pure Differential Privacy. The only reason that our algorithm requires approximate differential privacy is that we use truncated Laplace noise in line 19 of Algorithm 1. We use truncated Laplace noise so that if any leaf node is empty, then the noisy count will be 0 with probability 1 . To make the algorithm pure-DP, we modify line 19 to add Lap(2 /ϵ ) noise to the count c v , and in lines 20-21, we replace c v with 0 whenever the noised count is at most 10 ϵ · K log T . For the parameters in subsection 3.4, this is at most 1 · (log n ) O (1) .

̸

The difficulty, however, is that we need to speed up the runtime from T K , as done in the runtime analysis in subsection 3.3. Specifically, we need to ensure that few nodes are created (i.e., have c v = 0 ): we prove that, in expectation, this holds. In addition, we need to determine which nodes will have c v = 0 efficiently. If so, we only need to create these nodes in the preprocessing, which completes the proof.

First, we determine which nodes every data point p ∈ X is sent to, in the same fashion as in runtime analysis in subsection 3.3. Suppose there are n ′ ≤ n nonempty regions: then there are N := T K -n ′ empty regions. For each node v , we wish to sample c v ∼ Lap(2 /ϵ ) as the noise, and if c v > 10 ϵ · K log T , we keep the node v and store c v . Otherwise, we reset c v back to 0 , and do not need to store v .

Wenote that the probability of any fixed c v > 4 ϵ · K log T is 1 2 · e -( ϵ/ 2) · (10 /ϵ ) · K log T = e -5 K log T / 2 = T -5 K / 2 . Let α := T -5 K / 2 . We first consider the problem of sampling from Bin ( N,α ) , which represents the number of empty regions that will have c v > 0 . Note that P ( Bin ( N,α ) > 0) = 1 -(1 -T -5 K / 2) T K -n ′ ≤ T -4 K . By using a repeated squaring approach, we can compute this probability exactly in poly log( T K ) = poly log n time. Then, we can flip a coin with this probability, and with at most T -4 K failure probability, the Binomial equals 0 . Otherwise, need to compute the distribution Bin ( N,α ) conditioned on it being at least 1 . However, we can compute each term ( N r ) α r (1 -α ) N -r = P ( Bin ( N,α ) = r ) in time at most O ( N 2 ) , which thus means we can compute every P ( Bin ( N,α ) = r ) in time O ( N 3 ) . Hence, in the T -4 K ≤ N -4 probability event that Bin ( N,α ) > 0 , we can still draw from the conditional distribution in time O ( N 3 ) . Hence, in expected time poly log n , we can draw from Bin ( N,α ) .

In the case that Bin ( N,α ) = r > 0 , because every c v (for empty regions v ) has the same distribution, we can create the full tree, and then uniformly choose the subset of r regions which will have noisy count at least 10 ϵ · K log T . This takes time at most T 2 K . Wethen have to compute each c v ∼ Lap(2 /ϵ ) for the r choices of v where c v > 10 ϵ · K log T . However, by using the well-known memoryless property of exponential distributions, the distribution of each c v is just 10 ϵ · K log T + Expo (2 /ϵ ) . Hence, we can compute the full data structure in time O ( T 3 K ) , in this setting. This, however, only happens with at most T -4 K probability.

Alternatively, if r = 0 , we can avoid computing any c v when v is empty. The rest of the analysis in this case proceeds identically to subsection 3.3. Finally, in either case, the time to answer each query, in expectation, is still the same as in subsection 3.3.

Finally, we need to verify the accuracy of the algorithm. In expectation, each c v is off by at most O ( T log K/ϵ ) = (log n ) O (1) /ϵ , as the noise adds an expected O (1 /ϵ ) error, and the replacing of c v with 0 only happens when c v < O ( T log K/ϵ ) . The analysis of Lemma 3.3 proceeds identically, except that the first source of error replaces the log 1 δ term with a T log K = (log n ) O (1) term.

Finally, the embedding of arbitrary Euclidean data into the Euclidean sphere, as in Appendix A.2, proceeds the same way. Overall, Theorem 1.1 still holds when δ = 0 , except that the runtime of both the preprocessing and answering each query is in expectation, and the additive error removes the log(log m/δ ) term. (Indeed, the additional (log n ) O (1) factors in the runtime and additive error can be absorbed into the n o (1) term).

Large ϵ . We remark that our analysis extends to ϵ ≤ n o (1) in a straightfoward manner. We remark that in practice, because the probability ratio in differential privacy depends on e ϵ , it is highly impractical to use any ϵ significantly larger than Ω(log n ) , and therefore we believe that it is not interesting to analyze an algorithm beyond ϵ ≤ n o (1) .

To see why, note that for 1 ≤ ϵ ≤ n o (1) , any (1 / 2 , δ ) -DP algorithm is automatically ( ϵ, δ ) -DP. (This also holds in the δ = 0 case.) Note that the runtime bounds do not depend on ϵ , and the accuracy bounds are proportional to 1 /ϵ in the additive error. Therefore, if we allow the n ρ + o (1) term in the additive error to have a slightly larger o (1) in the exponent (i.e., up to n ρ + o (1) /ϵ ), the claim still holds until ϵ ≤ n o (1) for any o (1) .

Adaptive Queries. Here, we explain how we are able to improve Theorem 1.1 to handle adaptive queries.

We assume we are promised that all data points are in [ -R,R ] d , where R > r is some parameter. Let T represent the set of points ( x 1 , . . . , x d ) where every | x i | ≤ 2 R and x i is an integral multiple of r/d . Note that |T | ≤ O ( d · R/r ) d = e O ( d log( dR/r )) .

Now, consider having M = O ( d log( dR/r )) independent copies of the data structure, where each data structure is initialized with ( ϵ M , δ M ) -DP. To answer each query, each data structure computes its approximation, and the overall data structure outputs the mean of the M responses. By a standard Chernoff bound, and the accuracy bounds of Theorem 1.1, the output will be between (1 -o (1)) ·| X ∩ B 2 ( q, r ) |-n ρ + o (1) · log(log m · M/δ ) · log m ϵ/M and (1+ o (1)) ·| X ∩ B 2 ( q, cr ) | + n ρ + o (1) · log(log m · M/δ ) · log m ϵ/M , with e -Ω( M ) failure probability. In other words, the failure probability is e -Ω( M ) (assuming nonadaptive queries) and the additive error multiplies by at most O ( M log M ) . Moreover, each data structure is ( ϵ M , δ M ) -DP, so the overall algorithm, by basic composition, is ( ϵ, δ ) -DP. Finally, the runtime will only increase by a factor of M (both for preprocessing and answering queries), since we have to repeat the process for M data structures.

Now, we explain how we can deal with arbitrary adaptive queries. We can assume WLOG that every query is in the range [ -2 R, 2 R ] d , or else the algorithm can simply output 0 without sacrificing either privacy or accuracy. Next, we can assume WLOG that every query is in T , because any query q is within r/ √ d of some point q ′ ∈ T , and answering the query on q ′ will be accurate up to an additive error of r/ √ d = o ( r ) . Therefore, as long as the data structure is accurate for all of the queries in T , this is sufficient.

Thus, for each q ∈ T , the overall data structure will answer accurately with failure probability e -Ω( M ) . For M = O ( d log( dR/r )) , we have that |T | ≤ e Ω( M ) , so by a union bound, the data structure will answer any query q ∈ T accurately. Therefore, even adaptive queries can be answered accurately, since even any adaptive query can be assumed WLOG to be in T .

Peer reviews:
comments: SUMMARY:
This paper studies the problem of answering approximate range queries privately. The paper presents a data structure to answer queries privately that helps avoid dimension dependence in the additive error for utility but incurs a multiplicative factor. The paper also showcases how to efficiently implement the data structure.

STRENGTHS:
The paper is well structured, easy to follow with adequate discussion and comparison with prior work. 

The approach introduced provides a trade-off between additive error incurred and approximation factor for the query, which to the best of my knowledge is a novel contribution. The lower bound is for a restrictive case but understanding this dependence is a nice open problem.

WEAKNESSES:
The privacy guarantee is limited to $(\varepsilon, \delta)$-DP and not pure differential privacy. Some discussion on limitation or approach for extension to pure differential privacy might be helpful.

Also, it is not clear if the multiplicative factor in utility is necessary or inherent to this approach.

QUESTIONS:
Q1: Though $\varepsilon < 1$ is preferable for privacy, is there a barrier in extending Theorem 1.1 to case when $\varepsilon$ is $> 1$? 

Q2: How does the bound in Theorem 1.1 compare to other approaches for the case of $c=1$?

LIMITATIONS:
Discussed in weakness section. Since work is of theoretical nature, negative societal impact is not apparent.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2023-07-04

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe papers shows how to use a variant of LSH to approximately count the neighbors in an r-ball (r fixed apriori). It shows how to obtain a differentially private LSH sketch. It analyzes the algorithm’s theoretical properties.\n\nSTRENGTHS:\nThe technical contributions look solid and the analyses seem correct, e.g. using a weaker Markov inequality to handle non-independence of points. \n\nIt shows how to avoid an exponential dependence on the dimension. \n\nThe algorithm itself is fairly clear. \n\nThe paper makes an interesting claim that their method also leads to the most space efficient approximate nearest neighbor search.\n\nWEAKNESSES:\nThe presentation leaves a great deal to be desired. After the introduction, the exposition generally seems hastily arranged. Much of it is spent on the detailed analysis without offering much insight into either the algorithm or a high level overview of the proof strategy. The analysis skips details and crams equations into paragraphs which makes it harder to read. Since the proof is not short and many of the proofs are already in the appendix, it’d make sense to only outline the proof.\n\nThere is zero empirical validation. The result and algorithm are purely theoretical contributions.\n\nThere doesn’t seem to be any exposition supporting the claim that the paper yields the most space efficient known approx nearest neighbor search.\n\nQUESTIONS:\n1. The data structure created appears to be a (completely) random forest. What are the relationships between this work and differentially private random forests? \n\n2. Where do the 0.9 probability using Markov’s and 99% probability using Chernoff’s come from? I don’t see any other constants chosen that ensure these probabilities are reached.\n\n3. The extension to data not on a sphere is completely punted to the appendix. Can you give some insight on how this works, under what conditions it works, and what the impact to privacy is in the main text?\n\n4. The overview says that you “force” the LSH algorithm to create proper partitions. But where do you do this? Is it from the tree structure with depth? (as opposed to a stump-like structure) And doesn’t a single projection in LSH partition points into buckets so you can do fast lookups? Or by partition do you mean some fine partitioning?\n\n5.Can the structure answer a knn like query? I.e. What is the minimum radius r which contains k neighbors?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper provides a new polytime algorithm for differentially private approximate near neighbor counting---that is, privately counting the number of points inside l2 balls of fixed radius r, or more precisely a relaxation that may answer any value between the number of datapoints in B(x,r) and the number of datapoints in B(x, c*r) for parameter c. \n\nExisting algorithms incur either additive error that is at least a fixed polynomial in the number of points n or additive error that is logarithmic in n but grows exponentially in the dimension d. The algorithm in this paper incurs additive error that is n^(O(1/c^2)) where c is the parameter above, which can be made an arbitrarily small polynomial in n and has no dependence on d, at the cost of also introducing a small multiplicative error term.\n\n(As a side result, the paper also provides a lower bound showing that for L-infinity balls, it is not possible to have additive error n^(o(1)) for constant c. It poses the more immediately relevant lower bound question for L2 balls as an open question.)\n\nSTRENGTHS:\nThis is a nice result that improves on the additive accuracy of known results when both n and d are large. The algorithm is natural and interesting, and the paper is well-written and enjoyable.\n\nWEAKNESSES:\nIt's not clear whether the multiplicative approximation factor is necessary to get a good additive dependence on both n and d. (It's perfectly reasonable to trade off a small multiplicative factor for a better additive term, so this is a bit of a nitpick. However, it does mean that the result is incomparable rather than strictly better compared to prior work even in the regime when n and d are both large.)  \n\nRelatedly, the lower bound (in addition to being for l_\\infty rather than l_2) does not take into account the multiplicative approximation, so it's not clear whether it's possible to obtain additive error n^{o(1)} for constant c if we also allow a multiplicative error. \n\nMinor comments:\nThe abstract should mention the multiplicative aspect of the approximation.\n42: Maybe say explicitly \\ell_2 ball here, not just ball.\n69: The accuracy guarantee should only hold with high probability for the m queries q_1, ..., q_m, not for every q\\in \\mathbb{R}^d.\n128: I'm not positive, but I think one of the interval query results of BNSV15 has been improved in subsequent work to reduce the gap between the upper and lower bounds.\n200: It's technically not a partition, since some points are not sent to any child.\n\nQUESTIONS:\n1) Does the algorithm extend to other norm balls or other natural families of queries?\n\n2) Is there hope of removing the multiplicative approximation factor? Or else reason to believe that it is necessary?\n\n3) Do you think the analysis could go through for adaptive queries as well, or is it clear that the restriction to non-adaptive queries is necessary?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this work, the authors propose a differentially private data structure to approximately count the number of data points from within a dataset that lie within a certain small radius of a query point. \n\nThe preliminary data structure, intended for datasets that lie on a unit sphere, recursively splits the region into small caps. This is represented using a T-ary tree of height K. The leaf nodes of this tree maintain counts of all the points that lie within their corresponding regions. Privacy is obtained by perturbing these points using a truncated Laplacian mechanism. \n\nThe proofs follow from simple Gaussian concentration bounds and standard Laplacian mechanism.\n\nSTRENGTHS:\n- The problem is relevant and interesting. \n- The solution is simple and elegant and circumvents the drawbacks of prior works. \n- The authors provide a novel insight for using Locality sensitive hashing schemes for approximate counting.\n- The presentation is very clear and concise.\n\nWEAKNESSES:\n- The improved results only hold for small values of r which is fixed in advance and is not a part of the input.\n\nQUESTIONS:\n- Can you convert any data-independent LSH with LSH constant \\rho to construct an approximate near neighbor counting using a similar tree structure in a black-box fashion? For instance, each internal node will correspond to all the points from the parent node that fall within a certain region. The error estimates will be analyzed in a similar fashion and will be guaranteed by the LSH parameters. \nThe insight comes from the \\rho used in the theorem statement whose value is exactly the optimal LSH constant for data-independent schemes. \n\n- How about data-dependent LSH? The construction of Andoni & Razenshteyn provides a better LSH constant and does a recursive splitting that looks similar to the one used in this work.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-07"}