Repository: scipy/scipy

Issue:
Massive performance regression in cKDTree.query with L_inf distance in scipy 0.17
Hi,

While running some of my own code, I noticed a major slowdown in scipy 0.17 which I tracked to the cKDtree.query call with L_inf distance.
I did a git bisect and tracked the commit that caused the issue -- the commit is :
58a10c2688398ab440de8d6d7b5fba29671df182
Here is the example program that finishes in ~ 10 seconds before this commit and pretty much never finishes after it

```
import scipy.spatial.ckdtree
import numpy as np
arr1 = np.random.uniform(size=(6000000, 3))
tree2 = scipy.spatial.ckdtree.cKDTree(arr1)
distances, ind = tree2.query(arr1, 1, p=np.inf)
```

I also checked that if I change the distance from L_inf to L_2, then program finishes pretty quickly. I also tried to stop the program with gdb a few times to see where the code is mostly spending the time on and it seems to be here scipy/spatial/ckdtree/src/query.cxx:311. I also tried to dig into the problematic commit but I don't know enough about ckdtree internals to suggest a patch. 

Thanks, 
 Sergey

If relevant, I'm seeing this issue on a both systems that I tried  64bit RHEL6.7 with gcc 4.4 and Debian 7.9 with gcc-4.7. In all the cases it is python 2.7



Discussion:
cc @sturlamolden @rainwoodman 

The commit in question is the one that adds periodic space query. 

Line 311 was not in that commit, and there is no way cKDTree is spending most of its time on a `__builtin_prefetch`. 

I suspect a reproducible example might help.The OP does not seed the random number generator, are you comparing the runs on the same dataset? 

@ev-br 
The original example that I was preparing for the bug report had a fixed dataset, but then I discovered that the same problem happens on random datasets of similar size (~ 6 million points in 3D). And as far as I am aware the regression is easily reproducable. I've just tried the example that I gave on my laptop (i didn't check it before)(gcc 4.6 64bit Ubuntu 12.04), and with scipy-0.17 my test program didn't finish after 2 min, while with scipy-0.16.1 it finished in ~ 30 seconds.  
And obviously the problem persists if I stick np.random.seed(1) in the program -- so it is not related to a particularly weird point distribution.

@sturlamolden 
The fact that the line 311 wasn't part of the commit doesn't prove anything, as the periodic boundary commit did make several changes that specifically dealt with p=np.inf case (and there is no doubt that that the offending commit was 58a10c2 as I did git bisect). And if it helps here is the typical gdb backtrace that I encounter when try to stop the running process

```
#0  0x00007fd3c8969ad8 in point_point (k=0, y=0x7fd3be25ef58, 
    x=0x7fd3c3898e08, tree=<optimized out>)
    at scipy/spatial/ckdtree/src/distance.h:21
#1  distance_p (upperbound=-0, k=3, y=0x7fd3be25ef58, x=0x7fd3c3898e08, 
    tree=<optimized out>, p=<optimized out>)
    at scipy/spatial/ckdtree/src/distance.h:180
#2  query_single_point<BaseMinkowskiDistPinf<Dist1D> > (
    self=self@entry=0x7fd3c6dfa118, result_distances=0x7fd3b86d0528, 
    result_indices=0x7fd3b5909528, x=x@entry=0x7fd3be25ef58, k=k@entry=1, 
    eps=6.9439620697216583e-310, eps@entry=0, p=p@entry=inf, 
    distance_upper_bound=-0, distance_upper_bound@entry=inf)
    at scipy/spatial/ckdtree/src/query.cxx:311
#3  0x00007fd3c89728ee in query_knn (self=0x7fd3c6dfa118, dd=0x7fd3b86d0010, 
    ii=0x7fd3b5909010, xx=<optimized out>, n=6000000, k=1, eps=0, p=inf, 
    distance_upper_bound=inf) at scipy/spatial/ckdtree/src/query.cxx:511
```

...

The reason I mentioned the line 311 is that it seems to be in the part of code that is commented as /\* brute force */ which I thought probably points at the cause of the problem (just IMO)

Oh, sorry. In 0.17 L311 is the distance function, not the prefetch, which makes more sence. It did change to use C++ templates instead of a conditional branch on L_p. Replacing millions of conditional branches with nothing should not make the code slower. That is, unless there is something with your compiler that prevents it from inlining the template code. 

@sturlamolden No problem.
I think the compiler theory is also unlikely, given that I see the same issue with at least 3 different compilers (gcc 4.4, 4.6 4.7). But also I think a lot of "templatisation" of scipy.spatial-0.17 happened after the 58a10c2 commit so I would speculate that it is not the root cause of the problem.

It seems https://github.com/scipy/scipy/commit/58a10c2688398ab440de8d6d7b5fba29671df182 is slightly changing the logic for non-periodic queries, and for L_inf particularly. That might be the reason, but I don't know. We used templates to avoid this, but if not all of it is gone I will have to undo some of the changes made by @rainwoodman 

I feel sorry we didn't catch it with the benchmarks!

I wonder if this is as slow as in the current head. It should be fixable. I'll take a look!

I am looking too, but I am not sure what it might be yet. Perhaps it is related to the side distances (note the difference in handling for L_inf).

I don't think `point_point` is slow. It is more likely that it is called more often than it should, which points to something with the logics. Perhaps we a pushing too many nodes on the queue? 

Interestingly, the periodic case is way faster than the non-periodic case for p=inf!

It is likely in the logic. count_neighbors is not affected.

Line 385:

```
                npy_float64 tmp = x[split_dim] - split;
                if(NPY_LIKELY(p == 2)) {
                    inf2->side_distances()[split_dim] = tmp * tmp;
                } else {
                    inf2->side_distances()[split_dim] = std::pow(dabs(tmp), p);
                }

```


Resolving patch:
diff --git a/benchmarks/benchmarks/spatial.py b/benchmarks/benchmarks/spatial.py
--- a/benchmarks/benchmarks/spatial.py
+++ b/benchmarks/benchmarks/spatial.py
@@ -40,49 +40,47 @@ def time_build(self, mnr, cls_name):
         else:
             self.cls(self.data)
 
+LEAF_SIZES = [8, 128]
+BOX_SIZES = [None, 1.0]
 
 class Query(Benchmark):
     params = [
         [(3,10000,1000), (8,10000,1000), (16,10000,1000)],
-        ['KDTree', 'cKDTree', 'cKDTree_flat'],
+        [1, 2, np.inf],
+        BOX_SIZES, LEAF_SIZES,
     ]
-    param_names = ['(m, n, r)', 'class']
+    param_names = ['(m, n, r)', 'p', 'boxsize', 'leafsize']
 
     @staticmethod
-    def do_setup(self, mnr, cls_name):
-        self.cls = KDTree if cls_name == 'KDTree' else cKDTree
+    def do_setup(self, mnr, p, boxsize, leafsize):
         m, n, r = mnr
 
         np.random.seed(1234)
-        self.data = np.concatenate((np.random.randn(n//2,m),
-                                    np.random.randn(n-n//2,m)+np.ones(m)))
 
-        self.queries = np.concatenate((np.random.randn(r//2,m),
-                                       np.random.randn(r-r//2,m)+np.ones(m)))
+        self.data = np.random.uniform(size=(n, m))
+        self.queries = np.random.uniform(size=(r, m))
 
-        if cls_name == 'cKDTree_flat':
-            self.T = self.cls(self.data, leafsize=n)
-        else:
-            self.T = self.cls(self.data)
+        self.T = cKDTree(self.data, leafsize=leafsize, boxsize=boxsize)
 
-    def setup(self, mnr, cls_name):
-        Query.do_setup(self, mnr, cls_name)
+    def setup(self, mnr, p, boxsize, leafsize):
+        Query.do_setup(self, mnr, p, boxsize, leafsize)
 
-    def time_query(self, mnr, cls_name):
+    def time_query(self, mnr, p, boxsize, leafsize):
         """
         Querying kd-tree
         dim | # points | # queries |  KDTree  | cKDTree | flat cKDTree
         """
-        self.T.query(self.queries)
+        self.T.query(self.queries, p=p)
 
 
 class Radius(Benchmark):
     params = [
         [(3,10000,1000)],
+        [1, 2, np.inf],
         [0.2, 0.5],
-        ['KDTree', 'cKDTree', 'cKDTree_flat'],
+        BOX_SIZES, LEAF_SIZES,
     ]
-    param_names = ['(m, n, r)', 'probe radius', 'class']
+    param_names = ['(m, n, r)', 'p', 'probe radius', 'boxsize', 'leafsize']
 
     def __init__(self):
         self.time_query_pairs.__func__.params = list(self.params)
@@ -90,14 +88,14 @@ def __init__(self):
                                                     (8,1000,30),
                                                     (16,1000,30)]
 
-    def setup(self, mnr, probe_radius, cls_name):
-        Query.do_setup(self, mnr, cls_name)
+    def setup(self, mnr, p, probe_radius, boxsize, leafsize):
+        Query.do_setup(self, mnr, p, boxsize, leafsize)
 
-    def time_query_ball_point(self, mnr, probe_radius, cls_name):
-        self.T.query_ball_point(self.queries, probe_radius)
+    def time_query_ball_point(self, mnr, p, probe_radius, boxsize, leafsize):
+        self.T.query_ball_point(self.queries, probe_radius, p=p)
 
-    def time_query_pairs(self, mnr, probe_radius, cls_name):
-        self.T.query_pairs(probe_radius)
+    def time_query_pairs(self, mnr, p, probe_radius, boxsize, leafsize):
+        self.T.query_pairs(probe_radius, p=p)
 
 
 class Neighbors(Benchmark):
@@ -105,30 +103,27 @@ class Neighbors(Benchmark):
         [(3,1000,1000),
          (8,1000,1000),
          (16,1000,1000)],
+        [1, 2, np.inf],
         [0.2, 0.5],
-        ['KDTree', 'cKDTree'],
+        BOX_SIZES, LEAF_SIZES,
     ]
-    param_names = ['(m, n1, n2)', 'probe radius', 'class']
+    param_names = ['(m, n1, n2)', 'p', 'probe radius', 'boxsize', 'leafsize']
 
-    def setup(self, mn1n2, probe_radius, cls_str):
+    def setup(self, mn1n2, p, probe_radius, boxsize, leafsize):
         m, n1, n2 = mn1n2
 
-        cls = KDTree if cls_str == 'KDTree' else cKDTree
-
-        data1 = np.concatenate((np.random.randn(n1//2,m),
-                                np.random.randn(n1-n1//2,m)+np.ones(m)))
-        data2 = np.concatenate((np.random.randn(n2//2,m),
-                                np.random.randn(n2-n2//2,m)+np.ones(m)))
+        self.data1 = np.random.uniform(size=(n1, m))
+        self.data2 = np.random.uniform(size=(n2, m))
 
-        self.T1 = cls(data1)
-        self.T2 = cls(data2)
+        self.T1 = cKDTree(self.data1, boxsize=boxsize, leafsize=leafsize)
+        self.T2 = cKDTree(self.data2, boxsize=boxsize, leafsize=leafsize)
 
-    def time_sparse_distance_matrix(self, mn1n2, probe_radius, cls_str):
-        self.T1.sparse_distance_matrix(self.T2, probe_radius)
+    def time_sparse_distance_matrix(self, mn1n2, p, probe_radius, boxsize, leafsize):
+        self.T1.sparse_distance_matrix(self.T2, probe_radius, p=p)
 
-    def time_count_neighbors(self, mn1n2, probe_radius, cls_str):
+    def time_count_neighbors(self, mn1n2, p, probe_radius, boxsize, leafsize):
         """
         Count neighbors kd-tree
-        dim | # points T1 | # points T2 | probe radius |  KDTree  | cKDTree
+        dim | # points T1 | # points T2 | p | probe radius |  BoxSize | LeafSize
         """
-        self.T1.count_neighbors(self.T2, probe_radius)
+        self.T1.count_neighbors(self.T2, probe_radius, p=p)


Test patch: