SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Intuition for instance in Figure 7 The main reason why we construct a new example here is to handle the use of KD-tree to search for a good starting point. Implementation of KD-tree provided in [37] always randomly picks one of the top 5 dimensions with the largest variance as the splitting
Figure 7: Our constructed hard instance for SPTAG on the scale  $n = 10^6$ . The instance lives in a two-dimensional Euclidean space, and therefore has a constant doubling dimension. Black dots (solid or hollow) are points in the database, the red dot is the query point. Grids are only used to show the layout structure. See section C.1 for detailed description.
dimension, and divides the vertices into two halves at their mean. In Figure 7, because of the separation on the other 4 dimensions, our construction can make sure that KD-tree quickly gets to a subtree with vertices only in  $P$ . Then KD-tree will only horizontally split the chain from  $p_h$  to  $p_l$  or split via the other 4 dimensions. In the vertical axis, the coordinates for vertices in  $P$  and  $a$  are quite close, so KD-tree will assign them a low distance estimation based on only a horizontal split, resulting in KD-tree scanning all vertices on the chain before scanning other vertices outside of the set  $P$ . As long as we make the KD-tree select a vertex in  $P$  as the starting point, we can ensure (as in Figure 2) that GreedySearch will scan all vertices in  $P$  before going to  $M$  or  $P'$ .
C.2 More experiments on other popular nearest neighbor search algorithms
We further test the other 7 popular nearest neighbor search algorithms studied in the survey [37]. We use the same setting as in Section 4. We run each algorithm for 20 different data sizes  $n \in \{10^5, 2 \cdot 10^5, \dots, 2 \cdot 10^6\}$  using hard instances in Figure 4, Figure 2, or Figure 7 (introduced in Appendix C.1). We plot the Recall@5 rate for answering the query with queue length  $L$  equal to  $pn$  where the percentage  $p$  is enumerated from the set  $1\%, 2\%, \dots, 12\%, 15\%, 18\%, 20\%, 30\%, 40\%, 50\%$ .
NGT [20] We use NGT's implementation from the authors' GitHub repository [19]. We run NGT on the hard instance in Figure 2, using all default parameters as stated in GitHub'sREADME, except that we use the command "-i g" to generate only the graph index, because of our focus on graph-based nearest neighbor search algorithms. We use command "-p 1" to set the number of threads to 1. We run this experiment 10 times and report the average recall.
SSG [14] We use SSG implementation from the authors GitHub repository [13]. We run SSG on the hard instance in Figure 4, using parameters  $K = 200$ ,  $L = 200$ ,  $iter = 12$ ,  $S = 10$ ,  $R = 100$  (for building KNN graph)  $L = 100$ ,  $R = 50$ ,  $Angle = 60$  (constructing SSG). The parameters chosen here are copied from the author's selected parameters for data set SIFT1M [23], whose data size is close to ours. We run this experiment 10 times using different random seeds and report the average recall.
KGraph We use KGraph implementation due to [37], from the GitHub repository [38]. We run KGraph on the hard instance in Figure 4 using parameter  $K = 100$ ,  $L = 130$ ,  $iter = 12$ ,  $S = 20$ ,  $R = 50$ . Parameters here are copied from the authors' selected parameters used for their synthetic data set named "n_1000000", whose size is close to ours. We run this experiment 5 times using different random seeds and report the average recall.
Figure 8: Results for running NGT algorithm on the family of instances in Figure 2. The horizontal axis represents the data size  $n$ , in multiples of  $10^5$  points. The vertical axis represents the size of the search queue length  $L$  in terms of the percentage of the data size. Each pixel represents a query result, with its  $Recall@5 \in [0,1]$  mapping to the spectrum on the right. We run NGT algorithm 10 times and report the average recall rate.
DPG [27] We use DPG implementation due to [37], from the GitHub repository [38]. We run DPG on the hard instance in Figure 4 using parameter  $K = 100$ ,  $L = 100$ ,  $iter = 12$ ,  $S = 20$ ,  $R = 300$ . Parameters here are copied from the authors' selected parameters for their synthetic dataset named "n_1000000", whose size is close to ours. We run this experiment 10 times using different random seeds and report the average recall.
NSW [30] We use NSW's implementation due to [37], from the GitHub repository [38]. We run NSW on the hard instance in Figure 4 (with a different vertex permutation) using parameter  $\max\_m0 = 100$ ,  $ef\_construction = 400$ . Parameters here are copied from the author's selected parameters for their synthetic dataset named "n_1000000", whose data size is close to ours. We run this experiment 5 times using different random seeds and report the average recall.
SPTATG-KDT [7] We use SPTATG-KDT implementation due to [37], from the GitHub repository [38]. We run SPTATG-KDT on the hard instance in Figure 7 using parameters  $KDT\_number = 1$ ,  $TPT\_number = 16$ ,  $TPT\_leaf\_size = 1500$ ,  $scale = 2$ ,  $CEF = 1500$ . Parameters here are copied from the authors' selected parameters for their synthetic dataset named "n_1000000", whose size is close to ours. We run this experiment 5 times and report the average recall.
EFANNA [12] We use EFANNA's implementation due to [37], from the GitHub repository [38]. We run EFANNA on the hard instance in Figure 7 using parameter  $nTrees = 4$ ,  $mLevel = 8$ ,  $K = 80$ ,  $L = 140$ ,  $iter = 7$ ,  $S = 25$ ,  $R = 150$ . Parameters here are copied from the authors' selected parameters for their synthetic dataset named "n_1000000", whose size is close to ours. We run this experiment 10 times and report the average recall.
Experimental results for these 7 algorithms are plotted in Figure 8, Figure 9, and Figure 10. We can see that all algorithms achieve suboptimal Recall@5 rates until the queue length  $L$  is greater than  $10\%$  of the data size.
Figure 9: Results for running SSG, KGraph, DPG, NSW algorithms on the family of instances in Figure 4. The horizontal axis represents the data size  $n$ , in multiples of  $10^5$  points. The vertical axis represents the size of the search queue length  $L$  in terms of the percentage of the data size. Each pixel represents a query result, with its  $Recall@5 \in [0,1]$  mapping to the spectrum on the right. We run each algorithm 10 (or 5) times and report the average recall rate.
Figure 10: Results for running SPTAG-KDT and EFANNA algorithms on the family of instances in Figure 7. The horizontal axis represents the data size  $n$ , in multiples of  $10^5$  points. The vertical axis represents the size of the search queue length  $L$  in terms of the percentage of the data size. Each pixel represents a query result, with its  $Recall@5 \in [0,1]$  mapping to the spectrum on the right. We run each algorithm 10 (or 5) times and report the average recall rate.