SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We draw the instance in Figure 1. The whole instance can be embedded in a two dimensional grid using  $l_{1}$  distance. Note that in the figure, grids are drawn to label distances and highlight the layout structure. The vertex set  $V$  consists of a point set  $P$  and two single points  $p'$  and  $a$ . The point set  $P$  further consists of a  $\sqrt{n} \times \sqrt{n}$  (we assume that  $n$  is a perfect square) square grid with grid length  $\frac{0.5\epsilon}{\sqrt{n}}$  where each grid point is associated with a point. We denote the upper-rightmost point in  $P$  by  $p_{0}$ . Some important distances are  $D(p_{0}, p') = 2$ ,  $D(p', a) = \frac{2}{\alpha - 1}$ ,  $D(p_{0}, a) = \frac{2\alpha}{\alpha - 1}$ ,  $D(a, q) = 1 + \epsilon$ ,  $D(p_{0}, q) = \frac{2}{\alpha - 1} + 1 - \epsilon$ , where  $\epsilon < 0.01$ .
Lemma 3.7. Some properties regarding the graph  $G = (V, E)$  built on the instance in Figure 1 using the slow preprocessing version of DiskANN:
(1) For any  $p \in P$ ,  $(p, p') \in E$  and  $(p, a) \notin E$
(2) The subgraph of  $G$  induced by point set  $P \subseteq V$  is strongly connected.
(3) The starting vertex  $s$  is in  $P$ .
Proof. (1): According to the  $l_{1}$  distance, we can see that for any point  $p \in P$ ,  $D(p, p') > D(p_{0}, p') = 2$  and  $D(p, a) > D(p_{0}, a) = 2 + \frac{2}{\alpha - 1} = \frac{2\alpha}{\alpha - 1}$ , and  $D(p', a) = \frac{2}{\alpha - 1}$ . Therefore, in the procedure RobustPruning  $(p, V, \alpha)$ , the edge  $(p, p')$  remains and edge  $(p, a)$  is pruned.
(2): Because the point set  $P$  is a uniform grid, we know that for each vertex  $p$ , the distances between  $p$  and its four adjacent vertices in the grid are smaller than to all other vertices and must remain after RobustPruning $(p, V, \alpha)$ . Therefore the subgraph induced by the vertex set  $P$  in the final graph  $G$  is strongly connected.
(3): In the implementation of DiskANN, the starting point is the closest vertex to the centroid of the whole vertex set, which lies in  $P$ .
Proof of Theorem 3.6. Now we analyze the behavior of running GreedySearch  $(s,q,L)$  on the instance above on the graph constructed by DiskANN's slow version. We will show that the first  $n$  vertices entering the queue are all vertices in the set  $P$ . Additionally, the only neighbor of set  $P$ ,  $p'$ , is further from  $q$  than any vertex in the set  $P$ . If  $L \leqslant n$ , GreedySearch  $(s,q,L)$  terminates before getting to  $p'$  or  $a$ . Consequently, the nearest vertex returned by GreedySearch  $(s,q,L)$  has distance at least  $\frac{2}{\alpha - 1} + 1 - \epsilon$  from  $q$ , which is not a  $\frac{\alpha + 1}{\alpha - 1}$ -approximate nearest neighbor when  $\epsilon$  approaches 0.
The first vertex in the queue is  $s \in P$ . According to property (1) in Lemma 3.7, no vertex in  $P$  is connected to  $a$ , any vertex  $p \in P$  has distance  $D(p,q) < D(p',q)$  thus has higher priority to be scanned than  $p'$ , and the subset  $P$  is strongly connected, so in the first  $n$  steps (recall that  $|P| = n$ ), GreedySearch(s,q,L) will always scan vertices in  $P$ .
In our experiments, we test the performance of DiskANN, NSG, and HNSW, on two of our constructed hard instances. We run each algorithm on 20 different data sizes  $n \in \{10^5, 2 \cdot 10^5, \dots, 2 \cdot 10^6\}$ . Each data set consists of  $n$  points in the two-dimensional plane. We plot Recall@5 rates (Figures 3 and 5) and approximation ratios (Figure 6) for answering the query with queue length  $L$  equal to  $pn$  where the percentage  $p$  is enumerated from the set  $1\%, 2\%, \dots, 12\%, 15\%, 18\%, 20\%, 30\%, 40\%, 50\%$ . Note that for all graph-based nearest neighbor search algorithms, the value  $L$  lower bounds the number of vertices scanned (and therefore the running time) of the algorithm.
Our results show that, unlike on standard benchmarks, the algorithms scan at least  $10\%$  of the points in our instances before finding quality nearest neighbors. All experiments were ran on Google Cloud. Since our experiments only use combinatorial measures of algorithm complexity (i.e., the number of vertices searched), the results do not depend on the exact details of the computer architecture.
4.1 Hard instance for DiskANN
Though we provide  $\left(\frac{\alpha + 1}{\alpha - 1}\right)$ -approximate ratio upper bound for slow preprocessing version of DiskANN algorithm, in the experiments, we show that there exists a family of instances where the DiskANN algorithm with fast preprocessing cannot reach any top 5 nearest neighbor before scanning  $10\%$  of the vertices. We test our constructed instance using the authors' DiskANN code (fast preprocessing) on GitHub [22]. For a comparison, we also test our constructed instance on our implementation of the slow preprocessing DiskANN algorithm.
Figure 2: Our constructed hard instance for DiskANN for  $n = 10^6$ . The instance lives in a two-dimensional Euclidean space, and therefore has a constant doubling dimension. Black dots (solid or hollow) are points in the database, the red dot is the query point. Grids are only used to show the layout structure. See section 4.1 for detailed description and supplementary materials for its implementation.
In Figure 2, we draw our constructed instance for the DiskANN algorithm for  $n = 10^6$ . It is easy to mimic this instance for other data sizes. The parameters used in our experiment are  $R = 70$ ,  $L = 125$  (suggested in the original DiskANN paper [21], section 4.1, for in-memory search experiments),  $\alpha = 2$ , num Threads  $= 1$ .
In the left plot of Figure 3, we can see that in most cases, DiskANN (fast preprocessing version) cannot achieve non-zero Recall@5 unless scanning at least  $10\%$  of the vertex set. In the right plot of Figure 3, we can see that DiskANN (slow preprocessing version) generates very sparse graph data structure given our hard instance. Furthermore, after preprocessing, greedy search is very efficient: it finds the exact nearest neighbor in only 2 steps! Overall, it can be seen that, on the hard instances proposed in this paper, the greedy search procedure for query answering is highly efficient after slow preprocessing, and slow (linear time) after fast preprocessing.
Figure 3: Results for both variants of the DiskANN algorithm on the instance in Figure 2. Left figure (fast preprocessing DiskANN): the horizontal axis represents the data size  $n$ , in multiples of  $10^5$  points. The vertical axis represents the size of the search queue length  $L$  in terms of the percentage of the data size. Each pixel represents the average value of  $\text{Recall} @5 \in [0,1]$  over 10 runs of the algorithm. Note the sharp translation of the recall (from 0 to 1) for  $L \approx 10\%$ . Right figure (slow preprocessing DiskANN): as indicated by Theorem 3.4, slow preprocessing DiskANN constructs a very efficient data structure. In the experiment, it finds the exact nearest neighbor in at most 2 steps of greedy search, with  $L = 1$ . In the figure we plot the maximum degree of the constructed graph (on the vertical axis) vs. the data size  $n$  in multiples of  $10^4$  points (on the horizontal axis). The plot ends at  $n = 10^5$ , as the slow preprocessing algorithm takes several hundred hours for larger values of  $n$ .
Description of instance in Figure 2 The instance lives in a 2-dimensional plane under the  $l_{2}$  distance, so in what follows, we give the coordinates for each point, which defines the distances between all points. In Figure 2,  $n = 10^6$ , the vertex set  $V$  consists of three sets of points  $M, P$  and  $P'$ , of sizes  $0.8n, 0.1n, 0.1n$  respectively, and a single answer point  $a$ . Let  $l = 0.01 * n = 10^4$ .  $M$  is a  $\sqrt{|M|} \times \sqrt{|M|}$  square grid with grid side length 1 whose bottom-right corner is  $m = (-1.2l, 1.2l)$ .  $P$  is a  $\sqrt{|P|} \times \sqrt{|P|}$  square grid with grid side length 1 whose upper-right corner is  $p = (-l, 0)$ .  $P'$  is a  $\sqrt{|P'|} \times \sqrt{|P'|}$  square grid with grid side length 1 and whose bottom-left corner is  $p = (0, l)$ . (We assume that  $|M|$ ,  $|P|$ ,  $|P'|$  are perfect squares.) The query point is  $q = (-0.4l, 0)$ , whose nearest neighbor is  $a = (0, 0.1l)$ . Since our experiments measure  $\text{Recall}@5$ , we add another 4 points very close to  $a$ , which is not shown in the figure for simplicity.
Intuition First, the start point  $s$  should lie in the point set  $M$ . The three key properties we want to maintain in the construction of the graph are that (1)  $s$  is always connected to at least one vertex in the point sets  $P$  and  $P'$ . (2) Except for the points randomly connected to  $a$  at initialization, only the points in the set  $P'$  can still have edges to  $a$  at the end of the construction. (3) In the final graph, at least  $L$  vertices in set  $P$  are reachable from start point  $s$  without passing through any vertex not in  $P$ . If these three properties hold, GreedySearch( $s, q, L$ ) scans only the vertices in  $P$  (except for the start point  $s$ ) until it finds a vertex  $p_i \in P$  which is randomly connected to  $a$  at initialization. In expectation, this requires scanning  $\Omega(n/R)$  vertices, and DiskANN won't reach the nearest neighbor  $a$  before that. The actual running time of the code is even slower.
4.2 Hard instance for NSG and HNSW
A variant of the hard instance from the previous section works for NSG [15] and HNSW [28] algorithms as well. In Figure 4, we draw this instance for  $n = 10^6$ . It is easy to mimic this instance for other data sizes.
The parameters of NSG are as follows. We set  $K = 400$ ,  $L = 400$ ,  $iter = 12$ ,  $S = 15$ ,  $R = 100$  for EFANNA [12] to construct the KNN graph. We set  $L = 60$ ,  $R = 70$ ,  $C = 500$  for constructing NSG from KNN graph. These parameters were used by the authors when testing on GIST1M dataset, whose size is similar to ours. The parameters of HNSW are as follows.  $ef_{construction} = 200$  (as used in their example code),  $M = 64$  (maximal degree limit in their suggested range),  $num\_ threads = 1$ . We test our constructed instance using the authors' code available at [29] (for HNSW) and at [16]
Figure 4: Our constructed hard instance for NSG and HNSW algorithms for data size  $n = 10^6$ . The instance lives in a two-dimensional Euclidean space, and therefore has a constant doubling dimension. Black dots (solid or hollow) are points in the database, the red dot is the query point. Grids are only used to show the layout structure. See Section 4.2 for a detailed description and supplementary materials for its implementation.
Figure 5: Results for running NSG and HNSW algorithm on instance in Figure 4. The horizontal axis represents the data size  $n$ , in multiples of  $10^5$  points. The vertical axis represents the size of the search queue length  $L$  in terms of the percentage of the data size. Each pixel represents the average value of  $Recall@5 \in [0,1]$  over 10 runs of the algorithm. Since the algorithm is randomized, each run generates and uses a different random seed.
(for NSG). We note that both implementations are randomized, but their random seeds are hardwired into the code. To obtain more informative results, we generate and use different random seeds for each algorithm execution, and plot the average recall.
In Figure 5, we can see that, for most values of  $n$ , both NSG and HNSW algorithms cannot achieve good Recall@5 unless scanning at least  $10\%$  of the vertex set.
Description of instance in Figure 4 This instance is based on the last instance in Figure 2. The main difference is that we use a few chains of points to connect the three subsets  $M$ ,  $P$ , and  $P'$ . Specifically, on the dotted line, we add points uniformly spaced out, separated by a distance of 5 in the horizontal and/or vertical directions. For  $n = 10^6$  and  $l = 0.01n = 10^4$ , the instance consists of  $400(0.04\%)$  points on the diagonal (from  $m = (-1.2l, -1.2l)$  to  $(-l, -l)$ ),  $2000(0.2\%)$  on the horizontal line (from  $(-l, l)$  to  $p' = (0, l)$ ) and  $2000(0.2\%)$  points on the vertical line (from  $(-l, l)$  to  $p = (-l, 0)$ ). In total, there are 4400 new added points on the chain compared with the previous instance, occupying  $0.44\%$  of the vertex set.
Intuition for the instance in Figure 4 The new added vertex chains are there to make KNN graph connected for those nearest neighbor algorithms using KNN as part of their constructions. Thanks to these chains, most of the vertices in subset  $P$  and  $P'$  are reachable from the start point. The construction algorithm will only add edges from the vertices in the set  $P'$  to  $a$ , but not from  $P$ . Then,
GreedySearch on query  $q$  will first traverse the whole subset  $P$  before going to  $P'$ . Therefore, GreedySearch cannot get to the nearest neighbor if the queue length limit  $L$  is smaller than  $|P|$ .
4.3 Cross-comparisons
We also evaluated DiskANN on hard examples for NSG and HNSW, and vice versa. For  $n = 10^6$  the results are similar to the ones reported in earlier sections: none of the algorithms can achieve non-zero recall unless  $L$  exceeds  $10\%$  of the data set size. We did not experiment with other values of  $n$ , as two different families of instances are anyway helpful for other algorithms, as outlined in the next section and described in detail in supplementary material.
4.4 Evaluating approximation ratio of the algorithms
In addition to  $Recall@5$ , we also measure the average approximation ratio of DiskANN (fast preprocessing), NSG and HNSW on slightly modified hard instances from Figure 2 and 4. The modifications only involve scaling of the entire instance and bringing points  $a$  and  $q$  closer, to amplify the approximate ratio for other points. As per Figure 6, the average ratio is very high unless  $L \geqslant 0.1n$ . Note that we do not include the plot for DiskANN (slow preprocessing) here because, as shown in Figure 3, that algorithm (with  $L = 1$ ) can find the exact nearest neighbor in two steps.
Figure 6: Average approximation ratio results for running DiskANN (fast preprocessing), NSG, and HNSW algorithm on instances in Figure 2 and 4. The horizontal axis depicts the data size  $n$ , in multiples of  $10^5$ . The vertical axis depicts the size  $L$  of the search queue, as a ratio to  $n$ . Each pixel represents the average approximation ratio over 10 runs of the algorithm. Since the algorithms are randomized, each run generates and uses a different random seed.
4.5 Experiments on other popular approximate nearest neighbor search algorithms
We also tested other popular approximate nearest neighbor search algorithms covered in the survey [37]: NGT [20], SSG [14], KGraph [37], DPG [27], NSW [30], SPTAG-KDT [7] and EFANNA [12]. We ran them on our hard instances, with  $n$  ranging over  $10^5, 2 \cdot 10^5 \ldots 2 \cdot 10^6$  and the queue length  $L = 0.1n$ . The table depicts average Recall@5 rates over all values of  $n$  and 5 or 10 repetitions. One can see that all algorithms achieve sub-optimal recall. See the appendix for more details.
DiskANN	NSG	HNSW	NGT	SSG	KGraph	DPG	NSW	SPTAG-KDT	EFANNA
0.0	0.27	0.1	0.05	0.16	0.42	0.37	0.02	0.02	0.12
Table 1: Average Recall@5 for the 10 algorithms surveyed in [37], on our constructed instances.
In this paper we study the worst-case performance of popular graph-based nearest neighbor search algorithms. We demonstrate empirically that almost all of them suffer from linear query times on carefully constructed instances, despite being fast on benchmark data sets [37]. The exception is DiskANN with slow-preprocessing, for which we bound the approximation ratio and the running time. However, its super-linear preprocessing time makes it difficult to use for large data sets.
An important question raised by our work is whether there is a fast preprocessing algorithm, and a query answering procedure, that are empirically fast (e.g., as fast as for DiskANN) while having worst case query time and approximation guarantees. Another interesting direction is to investigate whether it is possible to replicate our findings for real data sets, using adversarially selected queries.
Acknowledgement: This work was supported by the Jacobs Presidential Fellowship, the NSF TRIPODS program (award DMS-2022448), Simons Investigator Award and MIT-IBM Watson AI Lab.
[36] Jun Wang, Wei Liu, Sanjiv Kumar, and Shih-Fu Chang. Learning to hash for indexing big data—a survey. Proceedings of the IEEE, 104(1):34-57, 2015.
[37] Mengzhao Wang, Xiaoliang Xu, Qiang Yue, and Yuxiang Wang. A comprehensive survey and experimental comparison of graph-based approximate nearest neighbor search. arXiv preprint arXiv:2101.12631, 2021.
[38] Mengzhao Wang, Xiaoliang Xu, Qiang Yue, and Yuxiang Wang. A comprehensive survey and experimental comparison of graph-based approximate nearest neighbor search. https://github.com/Lsyhprum/WEAVESS, 2021.
A DiskANN Algorithm
1: Input Vertex  $i$ , candidate neighbor set  $U$ , pruning parameter  $\alpha$ , degree limit  $R$  (default  $R$  is  $n$  if not given)
2: Result Update  $N_{out}(i)$ , the set of out-neighbors of  $i$
3:  $U \gets U \cup N_{out}(i)$
4:  $N_{out}(i)\gets \emptyset$
5: while  $U \neq \emptyset$  and  $|N_{out}(i)| < R$  do
6:  $v \gets \operatorname{argmin}_{v \in U} D(x_v, x_i)$
7:  $N_{out}(i)\gets N_{out}(i)\cup v$
8:  $U\gets U\backslash v$
9:  $U \gets \{v' \in U : D(x_v, x_{v'}) \cdot \alpha > D(x_i, x_{v'})\}$
Algorithm 1 RobustPruning  $(i,U,\alpha ,R)$
Algorithm 2 GreedySearch(s,q,L)
1: Input Graph  $G = (V,E)$  , seed  $s$  query point  $q$  , queue length limit  $L$    
2: Output visited vertex list  $U$    
3:  $A\gets \{s\}$    
4:  $U\gets \emptyset$    
5: while  $A\backslash U\neq \emptyset$  do   
6:  $v\gets \mathrm{argmin}_{v\in A\backslash U}D(x_v,q)$    
7:  $A\gets A\cup N_{out}(v)$    
8:  $U\gets U\cup v$    
9: if  $|A| > L$  then   
0:  $A\gets$  top  $L$  closest vertices to  $q$  in  $A$    
1: end if   
2: end while   
3: sort  $U$  in increasing distance from  $q$    
4: return  $U$
Algorithm 3 DiskANN indexing algorithm (with fast preprocessing)
1: Input Point set  $P = \{x_{1}\dots x_{n}\}$  , degree limit  $R$  , queue length  $L$    
2: Output A proximity graph  $G = (V,E)$  where  $V = \{1..n\}$  are associated with point sets  $P$    
3:  $G\gets$  randomly sample a  $R$  -regular graph on vertex set  $V = \{1..n\}$    
4:  $s\gets$  vertex for the point closest to the centroid of  $P$    
5: for  $k = 1$  to 2 do   
6:  $\sigma \gets$  a random permutation of [1..n]   
7: for  $i = 1$  to n do   
8:  $U\gets GreedySearch(s,x_{\sigma (i)},L)$    
9: RobustPruning(  $\sigma (i),U,\alpha ,R)$    
10: for vertex  $j$  in  $N_{out}(\sigma (i))$  do   
11:  $N_{out}(j)\gets N_{out}(j)\cup \sigma (i)$    
12: if  $|N_{out}(j)| > R$  then   
13: RobustPruning(j,  $N_{out}(j),\alpha ,R)$    
14: end if   
15: end for   
16: end for   
17: end for
Algorithm 4 DiskANN indexing algorithm (with slow preprocessing)
1: Input Vertex set  $P = \{x_{1}\dots x_{n}\}$ , parameters: degree limit  $R$   
2: Output A proximity graph  $G = (V,E)$  where  $V = \{1..n\}$  are associated with point sets  $P$ .  
3:  $s \gets$  vertex for the point closest to the centroid of  $P$   
4: for  $i = 1$  to  $n$  do  
5:  $N_{out}(i) \gets \text{RobustPruning}(i,V,\alpha,R)$   
6: end for
B Dependence on aspect ratio  $\triangle$
Consider the following 1-dimensional data set with  $n = 2k$  points located at  $\{x_{i}\}_{i=1}^{n}$ , where
$$
x _ {i} = \left\{ \begin{array}{l l} \alpha^ {i} & \text {f o r 1 \leqslant i \leqslant k} \\ 2 \alpha^ {k} + \alpha^ {k} \beta - \alpha^ {2 k + 1 - i} & \text {f o r k <   i \leqslant n} \end{array} \right.
$$
and  $\beta = \max \left(\frac{1}{\alpha - 1}, \alpha - 1\right)$ . This is a symmetric line starting from 0 to  $(2 + \beta)\alpha^k$ . Each point's distance toward the closer endpoint is  $\alpha$  times larger than that of the previous point.
Lemma B.1. The graph  $G = (V, E)$  built on the above instance using the slow preprocessing version of DiskANN satisfies the following properties:
(1) For any  $i \in [k + 1, n]$ ,  $(i, k) \in E$  and  $(i, j) \notin E$  for any  $j < k$ .
(2) For any  $j < i \leqslant k$ ,  $(i,j) \in E$  if and only if  $j = i - 1$
Since the  $x_{i}$ 's are symmetric, the same properties also hold in the other direction.
Proof. (1): For any  $i \in [k + 1, n]$ , we can check that no vertex  $j$  such that  $k < j < i$  can delete  $k$  from  $i$ 's neighborhood, because  $\frac{x_j - x_k}{x_i - x_k} > \frac{\alpha^k\beta}{\alpha^k\beta + \alpha^k} \geqslant \frac{1}{\alpha}$ . Thus, we have  $(i, k) \in E$ . Similarly, we have that  $k$  will delete any vertex  $j < k$  from  $i$ 's neighborhood because  $\frac{x_k - x_j}{x_i - x_j} \leqslant \frac{\alpha^k}{\alpha^k + \alpha^k\beta} \leqslant \frac{1}{\alpha}$ . These two inequalities use that  $\beta = \max(\frac{1}{\alpha - 1}, \alpha - 1)$
(2): For any  $i \in [1, k]$ ,  $x_{i-1}$  is the closest point on  $x_i$ 's left, so  $(i, i-1) \in E$ . Then, for any  $j < i-1$ ,  $j$  will be deleted from  $i$ 's neighbor by  $i-1$  because  $\frac{x_{i-1}-x_j}{x_i-x_j} < \frac{\alpha^{i-1}}{\alpha^i} = \frac{1}{\alpha}$ .
Proof of Theorem 3.5. Based on the graph properties in Lemma B.1, let us determine the length of the shortest path from a starting point  $s$  (selected arbitrarily by the DiskANN algorithm) to a constant approximate nearest neighbor of a given query  $q$ . We select our query  $q$  to be either 0 or  $2\alpha^k + \alpha^k\beta$ , i.e., one of the two endpoints of the data set, whichever is farther from  $x_s$ . To find an  $O(1)$ -approximate nearest neighbor of the query  $q$  within  $l$  steps of GreedySearch, there should be at least one path with less than  $l$  hops from  $s$  to  $q$ 's approximate nearest neighbor. WLOG, let's assume  $q = 0$  and  $s > k$ . By Property (1) of Lemma B.1, among  $\{1\dots k\}$ , the vertex  $k$  is the only neighbor of any vertex on the right of  $k$ . By Property (2) of Lemma B.1, for any vertex  $i \in [1,k]$ , its only neighbor on its left is  $i - 1$ . Therefore, it takes at least  $l \geqslant \Omega(n)$  and  $l \geqslant \Omega(\log \Delta)$  steps for slow preprocessing DiskANN with GreedySearch to reach any  $O(1)$ -approximate nearest neighbor in this constructed instance.
C.1 Hard instance for KD-Tree based nearest neighbor search algorithm
Some nearest neighbor search algorithms use KD-tree to find the entry point for greedy search. In this case, we design a hard instance where KD-tree cannot get good entry point close to the nearest neighbor. See Figure 7. We draw our constructed instance for  $n = 10^6$ . It is easy to mimic this instance for other data sizes.
Description of instance in Figure 7 Our instance has 6 dimensions, where the first two dimensions of the instance are similar to Figure 2 but with different choices of parameters. Again, the vertex set  $V$  consists of three sets of points  $M, P, P'$  (with size  $0.1n, 0.1n, 0.8n$  respectively) and a single answer point  $a$ :  $M$  is a  $\sqrt{|M|} \times \sqrt{|M|}$  square grid with unit length 1 and with bottom right corner  $m = (-10^9, 10^9)$ .  $P$  is a vertical chain of points consisting of unit intervals from  $p_l = (-10^9, -0.05n)$  to  $p_h = (-10^9, 0.05n)$ .  $P'$  is a  $\sqrt{|P'|} \times \sqrt{|P'|}$  square grid with unit length 1 whose bottom left corner is  $p' = (0, 10^9)$ . The answer point is  $a = (0, 3 * 10^8)$ . The query point is  $q = (-3 * 10^8, 0)$ . For a vertex  $v \in M \cup P' \cup \{a\}$ , each of  $v$ 's other 4 coordinates are sampled from a uniform distribution supported on  $[5 * 10^7, 6 * 10^7]$ . For a vertex  $v \in P$ , its other 4 coordinates are sampled from a uniform distribution supported on  $[10^7, 2 * 10^7]$ . And for point  $q$ , its other 4 coordinates are all 0. Note that though such choices of parameters are tailored to the implementation of KD-trees used in SPTAG and EFANNA in [37], some general ideas are reusable for constructing hard instances for other implementations.