PERF: releasing the GIL, #8882
State: closed

closes #8882
closes #10139 

This is now implemented for all groupbys with the factorization part as well (which is actually a big part fo the time)

```
In [2]: N = 1000000

In [3]: ngroups = 1000

In [4]: np.random.seed(1234)

In [5]: df = DataFrame({'key' : np.random.randint(0,ngroups,size=N),
   ...:                 'data' : np.random.randn(N) })
```

```
def f():
    df.groupby('key')['data'].sum()

# run consecutivily
def g2():
    for i in range(2):
        f()
def g4():
    for i in range(4):
        f()
def g8():
    for i in range(8):
        f()

# run in parallel
@test_parallel(num_threads=2)
def pg2():
    f()

@test_parallel(num_threads=4)
def pg4():
    f()

@test_parallel(num_threads=8)
def pg8():
    f()
```

So seeing a nice curve as far as addtl cores are used (as compared to some of my previous posts).

```
In [19]: df
Out[19]: 
   after  before   speedup
2   27.3    52.1  1.908425
4   44.8   105.0  2.343750
8   83.2   213.0  2.560096
```


Labels:
name: Performance
description: Memory or execution speed performance

Comments:
What was the deal with `NA` and `nan`.  These were Python objects?

apparently doing something

```
cdef ndarray[double_t] foo

foo = np.empty(10,dtype='float64')

foo[0] = np.nan
```

actually assigns a python object (and does inference and such), so that you MUST have the gil,
but casting to a double seems to work though (as its now c-level)

```
cdef double nan = <double> np.nan
with nogil:
    foo[0] = nan
```

You can reuse NPY_NAN cross-platform, e.g., https://github.com/scikit-image/scikit-image/blob/master/skimage/feature/_texture.pyx#L11

@ahojnnes thxs, I did change it.

I updated this. works for all groupbys now. So it gives a general 5-10% speedup on most operations (even single threaded), as there is slightly less generated code (in `nogil` it pretty much generates straight c code).

so we still need to hold the gil for the `.resize` operation, but its generally quite limited when this happens (e.g. resizing is a function of the number of uniques for example), but the cost of this is pretty low.

There IS a soln where I can not use numpy resize at ALL. But its quite a bit more c-code and IMHO not really worth it.

Cool.  I'll try to run a few different groupby operations with dask.dataframe sometime early next week and report back.  I'm curious what were the operations that needed to be released?  Was there a core set of functionality or was it a large volume of trivial work?

TL;DR.

So lots of 'boilerplate' changes in `generated.pyx`, which is basically all of the algorithms for groupby ops themselves (e.g. sum,mean). These were straightforward insertions of `with nogil`, just needed to avoid typing the `object` ones (which for the most part was easy; though a bit tricky on some non-groupby routines, e.g. see #10213).

However, the big speeds occur once I fixed the `hashtable.pyx` factorizer (e.g. the `get_labels` routine). The trivial soln actually produces a very odd perf issue (see [here](https://groups.google.com/forum/#!topic/cython-users/jtZ0FOGX2vU)), which is a bug / documentation issue.

In effect, you are using a `nogil` function that happens to have a `with gil` block WITHIN it, then the function is way slower than if you inline the exact same code.

I 'fixed' this issue by adding some additional c-structures to hold data (basically I changed the low-level implementation); these can be easily passed around; downside is that are a little 'messy' in the code department.

note to @shoyer I didn't actually bypass the realloc of memory (e.g. `.resize`) of numpy arrays. though I do have a soln to that (basically manage a memory block myself, then turning that into a numpy array at the very end (in `to_array`), easy enough, but was running into some compile issues, so dropped it in the end).

I think that this is awesome and am afraid of it going stale.  Is there something blocking it from being merged into master?

@mrocklin We were waiting to get the 0.16.2 release out first, which happened last Friday. At this point I don't think there are any blockers.

nothing blocking. I am going to update and see if I can hit some of #10213.

I have to do a bit of odd code-generation because for example you want to simultaneously support `nogil` (for numeric) and `gil` for `object`, but its a context manager (and NOT a function, which has some odd per characteristics). IOW, you have to worry about formatting, a bit annoying.

actually I do need to add a doc-note to the whatsnew (and maybe a small mention in the enhancingperf section as well).

Hrm, yes, unfortunate that you can't manipulate the lock directly

yep, would be nice to put in something along the lines of an `#IFDEF`, but in reality the block is actually doing 'work' (its only a couple of lines where it saves the current context and such), so maybe not easy/possible.

Thought I'd caress this PR with a gentle ping.  

_ping_

is that euphemism for harassment?

Kind and loving harassment, yes.

any commentary @jorisvandenbossche @shoyer @cpcloud @TomAugspurger 

going to have a followup for more GIL stuff. Its actually tricky making sure its doing (and you are measure exactly the right stuff)

Great job!!

Woohoo!

_Pulls from master, checks benchmarks._

@mrocklin Can we already look forward to a new awesome blogpost? :-)

Best I have to offer at the moment are the slowly growing [`dask.dataframe` docs](http://dask.pydata.org/en/latest/dataframe.html)

I should definitely check it out once I find some time!

Wow! Great job! 👍🏻